== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:51:49 (1789581109) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=0 llite.lustre-ffff958a85892800.statahead_max=32 llite.lustre-ffff958a85892800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=0 total: 100 open/close in 2.44 seconds: 41.02 ops/second llite.lustre-ffff958a85892800.statahead_max=0 101 real 0m2.536s user 0m0.003s sys 0m0.289s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff958a85892800.statahead_max=32 llite.lustre-ffff958a85892800.statahead_max=32 101 real 0m1.393s user 0m0.008s sys 0m0.155s 'ls -l' 100 files with statahead: 1 sec statahead total: 130 statahead wrong: 2 agl total: 125 list_total: 29 fname_total: 101 hit_total: 4053 miss_total: 1272 swrong=2 ewrong=2 - open/close 670 (time 1789581138.50 total 10.01 last 66.96) total: 900 open/close in 13.23 seconds: 68.02 ops/second llite.lustre-ffff958a85892800.statahead_max=0 1001 real 0m21.188s user 0m0.032s sys 0m2.717s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff958a85892800.statahead_max=32 llite.lustre-ffff958a85892800.statahead_max=32 1001 real 0m9.722s user 0m0.025s sys 0m1.307s 'ls -l' 1000 files with statahead: 10 sec statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 101 hit_total: 5052 miss_total: 1273 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff958a85892800.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 101 hit_total: 5052 miss_total: 1273 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats= snapshot_time: 1789581203.055404860 secs.nsecs start_time: 1789581115.075738544 secs.nsecs elapsed_time: 87.979666316 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 827 (time 1789581215.98 total 10.00 last 82.67) - open/close 1524 (time 1789581226.00 total 20.02 last 69.57) - open/close 2328 (time 1789581236.01 total 30.03 last 80.32) - open/close 3064 (time 1789581246.02 total 40.04 last 73.54) - open/close 3634 (time 1789581256.04 total 50.06 last 56.89) - open/close 4371 (time 1789581266.05 total 60.07 last 73.59) - open/close 5187 (time 1789581276.07 total 70.09 last 81.46) - open/close 6030 (time 1789581286.07 total 80.09 last 84.28) - open/close 6708 (time 1789581296.08 total 90.11 last 67.71) - open/close 7512 (time 1789581306.09 total 100.12 last 80.33) - open/close 8219 (time 1789581316.11 total 110.13 last 70.57) - open/close 8986 (time 1789581326.12 total 120.14 last 76.64) - open/close 9728 (time 1789581336.13 total 130.15 last 74.12) total: 10000 open/close in 133.98 seconds: 74.64 ops/second llite.lustre-ffff958a85892800.statahead_batch_max=0 llite.lustre-ffff958a85892800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.385s user 0m0.305s sys 0m14.752s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff958a85892800.statahead_batch_max=32 llite.lustre-ffff958a85892800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.864s user 0m0.272s sys 0m13.336s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats= snapshot_time: 1789581519.739451041 secs.nsecs start_time: 1789581407.281134842 secs.nsecs elapsed_time: 112.458316199 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=0 llite.lustre-ffff958a85892800.statahead_max=2048 llite.lustre-ffff958a85892800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=0 total: 100 open/close in 2.29 seconds: 43.69 ops/second llite.lustre-ffff958a85892800.statahead_max=0 101 real 0m2.171s user 0m0.005s sys 0m0.285s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff958a85892800.statahead_max=2048 llite.lustre-ffff958a85892800.statahead_max=2048 101 real 0m0.936s user 0m0.006s sys 0m0.148s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 741 (time 1789581850.71 total 10.01 last 74.04) total: 900 open/close in 12.34 seconds: 72.91 ops/second llite.lustre-ffff958a85892800.statahead_max=0 1001 real 0m20.079s user 0m0.026s sys 0m2.620s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff958a85892800.statahead_max=2048 llite.lustre-ffff958a85892800.statahead_max=2048 1001 real 0m6.850s user 0m0.030s sys 0m1.378s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff958a85892800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats= snapshot_time: 1789581914.660417226 secs.nsecs start_time: 1789581522.014925922 secs.nsecs elapsed_time: 392.645491304 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 1 10 30 32: 2 20 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 686 (time 1789581927.70 total 10.00 last 68.60) - open/close 1428 (time 1789581937.72 total 20.01 last 74.10) - open/close 2182 (time 1789581947.72 total 30.02 last 75.35) - open/close 2927 (time 1789581957.73 total 40.03 last 74.44) - open/close 3674 (time 1789581967.73 total 50.03 last 74.69) - open/close 4457 (time 1789581977.76 total 60.05 last 78.10) - open/close 5254 (time 1789581987.76 total 70.06 last 79.65) - open/close 6010 (time 1789581997.76 total 80.06 last 75.58) - open/close 6688 (time 1789582007.84 total 90.13 last 67.32) - open/close 7305 (time 1789582017.84 total 100.14 last 61.67) - open/close 8034 (time 1789582027.84 total 110.14 last 72.89) - open/close 8738 (time 1789582037.84 total 120.14 last 70.39) - open/close 9465 (time 1789582047.85 total 130.15 last 72.66) total: 10000 open/close in 137.14 seconds: 72.92 ops/second llite.lustre-ffff958a85892800.statahead_batch_max=0 llite.lustre-ffff958a85892800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.690s user 0m0.279s sys 0m14.672s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff958a85892800.statahead_batch_max=256 llite.lustre-ffff958a85892800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff958a85892800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.139s user 0m0.266s sys 0m13.970s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff958a85892800.batch_stats= snapshot_time: 1789582183.791679346 secs.nsecs start_time: 1789582126.437067531 secs.nsecs elapsed_time: 57.354611815 secs.nsecs subreqs per batch batches %% cum %% 1: 75 39 39 2: 19 10 49 4: 16 8 58 8: 11 5 64 16: 13 6 70 32: 12 6 77 64: 6 3 80 128: 2 1 81 256: 35 18 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff958a85892800.statahead_batch_max=64 llite.lustre-ffff958a85892800.statahead_max=128