== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:07:45 (1788516465) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=0 llite.lustre-ffff953c43ec4800.statahead_max=32 llite.lustre-ffff953c43ec4800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=0 total: 100 open/close in 2.78 seconds: 36.00 ops/second llite.lustre-ffff953c43ec4800.statahead_max=0 101 real 0m2.661s user 0m0.004s sys 0m0.312s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff953c43ec4800.statahead_max=32 llite.lustre-ffff953c43ec4800.statahead_max=32 101 real 0m1.141s user 0m0.006s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 152 statahead wrong: 2 agl total: 146 list_total: 30 fname_total: 104 hit_total: 3549 miss_total: 1625 swrong=2 ewrong=2 - open/close 692 (time 1788516497.15 total 10.00 last 69.19) total: 900 open/close in 13.09 seconds: 68.74 ops/second llite.lustre-ffff953c43ec4800.statahead_max=0 1001 real 0m18.806s user 0m0.038s sys 0m2.786s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff953c43ec4800.statahead_max=32 llite.lustre-ffff953c43ec4800.statahead_max=32 1001 real 0m9.869s user 0m0.030s sys 0m1.322s 'ls -l' 1000 files with statahead: 9 sec statahead total: 153 statahead wrong: 2 agl total: 147 list_total: 31 fname_total: 104 hit_total: 4547 miss_total: 1627 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff953c43ec4800.statahead_stats= statahead total: 153 statahead wrong: 2 agl total: 147 list_total: 31 fname_total: 104 hit_total: 4547 miss_total: 1627 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats= snapshot_time: 1788516560.850817811 secs.nsecs start_time: 1788516472.050844435 secs.nsecs elapsed_time: 88.799973376 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 707 (time 1788516573.26 total 10.00 last 70.70) - open/close 1402 (time 1788516583.27 total 20.01 last 69.45) - open/close 2132 (time 1788516593.27 total 30.01 last 72.99) - open/close 2819 (time 1788516603.27 total 40.01 last 68.68) - open/close 3402 (time 1788516613.28 total 50.02 last 58.25) - open/close 4063 (time 1788516623.28 total 60.02 last 66.09) - open/close 4840 (time 1788516633.29 total 70.03 last 77.64) - open/close 5697 (time 1788516643.29 total 80.03 last 85.70) - open/close 6525 (time 1788516653.29 total 90.04 last 82.76) - open/close 7305 (time 1788516663.30 total 100.05 last 77.93) - open/close 8118 (time 1788516673.32 total 110.06 last 81.15) - open/close 8850 (time 1788516683.33 total 120.07 last 73.12) - open/close 9548 (time 1788516693.34 total 130.08 last 69.76) total: 10000 open/close in 136.00 seconds: 73.53 ops/second llite.lustre-ffff953c43ec4800.statahead_batch_max=0 llite.lustre-ffff953c43ec4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.323s user 0m0.331s sys 0m13.389s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff953c43ec4800.statahead_batch_max=32 llite.lustre-ffff953c43ec4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m38.616s user 0m0.311s sys 0m13.161s batched statahead RPC= hit count=9970 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats= snapshot_time: 1788516880.793160491 secs.nsecs start_time: 1788516772.634127128 secs.nsecs elapsed_time: 108.159033363 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=0 llite.lustre-ffff953c43ec4800.statahead_max=2048 llite.lustre-ffff953c43ec4800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=0 total: 100 open/close in 3.08 seconds: 32.45 ops/second llite.lustre-ffff953c43ec4800.statahead_max=0 101 real 0m2.321s user 0m0.001s sys 0m0.302s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff953c43ec4800.statahead_max=2048 llite.lustre-ffff953c43ec4800.statahead_max=2048 101 real 0m0.915s user 0m0.005s sys 0m0.149s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10069 miss_total: 31 swrong=0 ewrong=0 - open/close 715 (time 1788517197.44 total 10.00 last 71.49) total: 900 open/close in 12.66 seconds: 71.11 ops/second llite.lustre-ffff953c43ec4800.statahead_max=0 1001 real 0m22.267s user 0m0.034s sys 0m2.811s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff953c43ec4800.statahead_max=2048 llite.lustre-ffff953c43ec4800.statahead_max=2048 1001 real 0m6.714s user 0m0.021s sys 0m1.315s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11068 miss_total: 32 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff953c43ec4800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11068 miss_total: 32 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats= snapshot_time: 1788517265.281681445 secs.nsecs start_time: 1788516883.019953505 secs.nsecs elapsed_time: 382.261727940 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 692 (time 1788517278.80 total 10.01 last 69.12) - open/close 1305 (time 1788517288.81 total 20.02 last 61.26) - open/close 1988 (time 1788517298.82 total 30.02 last 68.27) - open/close 2771 (time 1788517308.84 total 40.04 last 78.15) - open/close 3568 (time 1788517318.85 total 50.05 last 79.62) - open/close 4281 (time 1788517328.86 total 60.07 last 71.19) - open/close 5087 (time 1788517338.86 total 70.07 last 80.57) - open/close 5885 (time 1788517348.88 total 80.08 last 79.69) - open/close 6708 (time 1788517358.89 total 90.09 last 82.24) - open/close 7561 (time 1788517368.89 total 100.09 last 85.30) - open/close 8394 (time 1788517378.89 total 110.09 last 83.29) - open/close 9225 (time 1788517388.90 total 120.11 last 82.97) - open/close 9997 (time 1788517398.91 total 130.12 last 77.15) total: 10000 open/close in 130.15 seconds: 76.83 ops/second llite.lustre-ffff953c43ec4800.statahead_batch_max=0 llite.lustre-ffff953c43ec4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.821s user 0m0.312s sys 0m13.461s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff953c43ec4800.statahead_batch_max=256 llite.lustre-ffff953c43ec4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff953c43ec4800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.973s user 0m0.341s sys 0m14.088s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff953c43ec4800.batch_stats= snapshot_time: 1788517536.933772435 secs.nsecs start_time: 1788517467.038804320 secs.nsecs elapsed_time: 69.894968115 secs.nsecs subreqs per batch batches %% cum %% 1: 113 38 38 2: 45 15 54 4: 32 10 65 8: 28 9 74 16: 22 7 82 32: 12 4 86 64: 5 1 88 128: 1 0 88 256: 34 11 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff953c43ec4800.statahead_batch_max=64 llite.lustre-ffff953c43ec4800.statahead_max=128