== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:02:16 (1788534136) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=0 llite.lustre-ffff8b968638e000.statahead_max=32 llite.lustre-ffff8b968638e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=0 total: 100 open/close in 3.81 seconds: 26.26 ops/second llite.lustre-ffff8b968638e000.statahead_max=0 101 real 0m4.173s user 0m0.004s sys 0m0.335s 'ls -l' 100 files without statahead: 5 sec llite.lustre-ffff8b968638e000.statahead_max=32 llite.lustre-ffff8b968638e000.statahead_max=32 101 real 0m1.622s user 0m0.006s sys 0m0.168s 'ls -l' 100 files with statahead: 2 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 30 fname_total: 79 hit_total: 3981 miss_total: 1309 swrong=2 ewrong=2 - open/close 682 (time 1788534169.66 total 10.00 last 68.20) total: 900 open/close in 13.82 seconds: 65.14 ops/second llite.lustre-ffff8b968638e000.statahead_max=0 1001 real 0m25.274s user 0m0.037s sys 0m3.234s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff8b968638e000.statahead_max=32 llite.lustre-ffff8b968638e000.statahead_max=32 1001 real 0m10.918s user 0m0.026s sys 0m1.316s 'ls -l' 1000 files with statahead: 11 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 79 hit_total: 4977 miss_total: 1313 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff8b968638e000.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 79 hit_total: 4977 miss_total: 1313 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats= snapshot_time: 1788534247.672658644 secs.nsecs start_time: 1788534141.618752437 secs.nsecs elapsed_time: 106.053906207 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 630 (time 1788534260.10 total 10.00 last 62.98) - open/close 1297 (time 1788534270.11 total 20.01 last 66.66) - open/close 1961 (time 1788534280.11 total 30.02 last 66.34) - open/close 2639 (time 1788534290.12 total 40.03 last 67.75) - open/close 3319 (time 1788534300.13 total 50.03 last 67.98) - open/close 4056 (time 1788534310.15 total 60.05 last 73.54) - open/close 4683 (time 1788534320.15 total 70.06 last 62.67) - open/close 5285 (time 1788534330.16 total 80.06 last 60.16) - open/close 5952 (time 1788534340.17 total 90.07 last 66.62) - open/close 6629 (time 1788534350.18 total 100.08 last 67.63) - open/close 7333 (time 1788534360.19 total 110.09 last 70.35) - open/close 7963 (time 1788534370.19 total 120.09 last 62.98) - open/close 8548 (time 1788534380.20 total 130.11 last 58.41) - open/close 9195 (time 1788534390.21 total 140.12 last 64.64) - open/close 9866 (time 1788534400.22 total 150.12 last 67.07) total: 10000 open/close in 152.20 seconds: 65.70 ops/second llite.lustre-ffff8b968638e000.statahead_batch_max=0 llite.lustre-ffff8b968638e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.603s user 0m0.358s sys 0m14.502s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b968638e000.statahead_batch_max=32 llite.lustre-ffff8b968638e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m43.282s user 0m0.316s sys 0m13.492s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats= snapshot_time: 1788534586.901222924 secs.nsecs start_time: 1788534472.089330721 secs.nsecs elapsed_time: 114.811892203 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=0 llite.lustre-ffff8b968638e000.statahead_max=2048 llite.lustre-ffff8b968638e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=0 total: 100 open/close in 3.23 seconds: 30.92 ops/second llite.lustre-ffff8b968638e000.statahead_max=0 101 real 0m2.530s user 0m0.005s sys 0m0.294s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b968638e000.statahead_max=2048 llite.lustre-ffff8b968638e000.statahead_max=2048 101 real 0m0.994s user 0m0.004s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 614 (time 1788534943.77 total 10.00 last 61.39) total: 900 open/close in 14.38 seconds: 62.59 ops/second llite.lustre-ffff8b968638e000.statahead_max=0 1001 real 0m22.489s user 0m0.036s sys 0m2.851s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8b968638e000.statahead_max=2048 llite.lustre-ffff8b968638e000.statahead_max=2048 1001 real 0m7.983s user 0m0.027s sys 0m1.496s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8b968638e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats= snapshot_time: 1788535019.384026090 secs.nsecs start_time: 1788534589.311672412 secs.nsecs elapsed_time: 430.072353678 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 606 (time 1788535032.37 total 10.01 last 60.53) - open/close 1197 (time 1788535042.38 total 20.01 last 59.08) - open/close 1801 (time 1788535052.38 total 30.02 last 60.39) - open/close 2437 (time 1788535062.38 total 40.02 last 63.59) - open/close 3063 (time 1788535072.38 total 50.02 last 62.59) - open/close 3673 (time 1788535082.40 total 60.04 last 60.88) - open/close 4326 (time 1788535092.41 total 70.05 last 65.24) - open/close 4963 (time 1788535102.42 total 80.06 last 63.64) - open/close 5539 (time 1788535112.44 total 90.08 last 57.49) - open/close 6079 (time 1788535122.44 total 100.08 last 53.98) - open/close 6698 (time 1788535132.45 total 110.09 last 61.85) - open/close 7283 (time 1788535142.47 total 120.10 last 58.41) - open/close 7805 (time 1788535152.50 total 130.13 last 52.03) - open/close 8378 (time 1788535162.52 total 140.16 last 57.18) - open/close 9067 (time 1788535172.54 total 150.17 last 68.78) - open/close 9692 (time 1788535182.56 total 160.19 last 62.37) total: 10000 open/close in 165.22 seconds: 60.52 ops/second llite.lustre-ffff8b968638e000.statahead_batch_max=0 llite.lustre-ffff8b968638e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.919s user 0m0.333s sys 0m14.321s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b968638e000.statahead_batch_max=256 llite.lustre-ffff8b968638e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b968638e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.803s user 0m0.316s sys 0m15.002s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b968638e000.batch_stats= snapshot_time: 1788535328.345128905 secs.nsecs start_time: 1788535265.368405904 secs.nsecs elapsed_time: 62.976723001 secs.nsecs subreqs per batch batches %% cum %% 1: 110 41 41 2: 44 16 57 4: 29 10 68 8: 16 6 74 16: 10 3 78 32: 17 6 84 64: 4 1 86 128: 1 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b968638e000.statahead_batch_max=64 llite.lustre-ffff8b968638e000.statahead_max=128