== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:41:15 (1784943675) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=0 llite.lustre-ffff9a5b0346f000.statahead_max=32 llite.lustre-ffff9a5b0346f000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=0 total: 100 open/close in 1.86 seconds: 53.84 ops/second llite.lustre-ffff9a5b0346f000.statahead_max=0 101 real 0m1.809s user 0m0.004s sys 0m0.307s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9a5b0346f000.statahead_max=32 llite.lustre-ffff9a5b0346f000.statahead_max=32 101 real 0m0.868s user 0m0.008s sys 0m0.199s 'ls -l' 100 files with statahead: 1 sec statahead total: 97 statahead wrong: 2 agl total: 92 list_total: 30 fname_total: 67 hit_total: 3540 miss_total: 1697 swrong=2 ewrong=2 total: 900 open/close in 8.84 seconds: 101.76 ops/second llite.lustre-ffff9a5b0346f000.statahead_max=0 1001 real 0m15.918s user 0m0.032s sys 0m2.984s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff9a5b0346f000.statahead_max=32 llite.lustre-ffff9a5b0346f000.statahead_max=32 1001 real 0m7.445s user 0m0.018s sys 0m1.464s 'ls -l' 1000 files with statahead: 8 sec statahead total: 98 statahead wrong: 2 agl total: 93 list_total: 31 fname_total: 67 hit_total: 4537 miss_total: 1700 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff9a5b0346f000.statahead_stats= statahead total: 98 statahead wrong: 2 agl total: 93 list_total: 31 fname_total: 67 hit_total: 4537 miss_total: 1700 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats= snapshot_time: 1784943743.261645034 secs.nsecs start_time: 1784943679.299575012 secs.nsecs elapsed_time: 63.962070022 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 1075 (time 1784943754.93 total 10.01 last 107.35) - open/close 2211 (time 1784943764.94 total 20.02 last 113.54) - open/close 3290 (time 1784943774.95 total 30.02 last 107.84) - open/close 4224 (time 1784943784.96 total 40.04 last 93.30) - open/close 5054 (time 1784943794.96 total 50.04 last 82.96) - open/close 6150 (time 1784943804.97 total 60.05 last 109.52) - open/close 7267 (time 1784943814.98 total 70.06 last 111.60) - open/close 8423 (time 1784943824.98 total 80.06 last 115.58) - open/close 9617 (time 1784943834.98 total 90.06 last 119.33) total: 10000 open/close in 93.39 seconds: 107.07 ops/second llite.lustre-ffff9a5b0346f000.statahead_batch_max=0 llite.lustre-ffff9a5b0346f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.131s user 0m0.281s sys 0m17.682s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a5b0346f000.statahead_batch_max=32 llite.lustre-ffff9a5b0346f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.711s user 0m0.258s sys 0m14.969s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats= snapshot_time: 1784943971.023203437 secs.nsecs start_time: 1784943888.955371198 secs.nsecs elapsed_time: 82.067832239 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=0 llite.lustre-ffff9a5b0346f000.statahead_max=2048 llite.lustre-ffff9a5b0346f000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=0 total: 100 open/close in 1.96 seconds: 50.96 ops/second llite.lustre-ffff9a5b0346f000.statahead_max=0 101 real 0m1.967s user 0m0.000s sys 0m0.310s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9a5b0346f000.statahead_max=2048 llite.lustre-ffff9a5b0346f000.statahead_max=2048 101 real 0m0.597s user 0m0.002s sys 0m0.154s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 total: 900 open/close in 8.16 seconds: 110.31 ops/second llite.lustre-ffff9a5b0346f000.statahead_max=0 1001 real 0m15.949s user 0m0.032s sys 0m2.945s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff9a5b0346f000.statahead_max=2048 llite.lustre-ffff9a5b0346f000.statahead_max=2048 1001 real 0m4.819s user 0m0.026s sys 0m1.386s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff9a5b0346f000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats= snapshot_time: 1784944256.859330182 secs.nsecs start_time: 1784943972.521928435 secs.nsecs elapsed_time: 284.337401747 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 1 10 60 128: 0 0 60 256: 4 40 100 - open/close 1053 (time 1784944268.55 total 10.00 last 105.29) - open/close 2161 (time 1784944278.56 total 20.01 last 110.74) - open/close 3239 (time 1784944288.57 total 30.01 last 107.70) - open/close 4352 (time 1784944298.58 total 40.02 last 111.21) - open/close 5356 (time 1784944308.58 total 50.03 last 100.34) - open/close 6355 (time 1784944318.58 total 60.03 last 99.89) - open/close 7467 (time 1784944328.59 total 70.03 last 111.15) - open/close 8502 (time 1784944338.61 total 80.06 last 103.24) - open/close 9418 (time 1784944348.62 total 90.07 last 91.52) total: 10000 open/close in 96.22 seconds: 103.93 ops/second llite.lustre-ffff9a5b0346f000.statahead_batch_max=0 llite.lustre-ffff9a5b0346f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.518s user 0m0.303s sys 0m17.998s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a5b0346f000.statahead_batch_max=256 llite.lustre-ffff9a5b0346f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.580s user 0m0.256s sys 0m16.843s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9a5b0346f000.batch_stats= snapshot_time: 1784944451.100066850 secs.nsecs start_time: 1784944404.826956612 secs.nsecs elapsed_time: 46.273110238 secs.nsecs subreqs per batch batches %% cum %% 1: 189 55 55 2: 38 11 66 4: 32 9 75 8: 13 3 79 16: 19 5 84 32: 12 3 88 64: 5 1 89 128: 0 0 89 256: 35 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9a5b0346f000.statahead_batch_max=64 llite.lustre-ffff9a5b0346f000.statahead_max=128