== sanity test 123ad: Verify batching statahead works correctly ========================================================== 08:16:57 (1790684217) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=0 llite.lustre-ffff91da43d57800.statahead_max=32 llite.lustre-ffff91da43d57800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=0 total: 100 open/close in 3.60 seconds: 27.78 ops/second llite.lustre-ffff91da43d57800.statahead_max=0 101 real 0m3.575s user 0m0.007s sys 0m0.328s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91da43d57800.statahead_max=32 llite.lustre-ffff91da43d57800.statahead_max=32 101 real 0m1.761s user 0m0.005s sys 0m0.202s 'ls -l' 100 files with statahead: 2 sec statahead total: 127 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 77 hit_total: 3823 miss_total: 1354 swrong=2 ewrong=2 - open/close 385 (time 1790684252.93 total 10.01 last 38.48) - open/close 830 (time 1790684262.93 total 20.01 last 44.48) total: 900 open/close in 21.41 seconds: 42.04 ops/second llite.lustre-ffff91da43d57800.statahead_max=0 1001 real 0m25.037s user 0m0.034s sys 0m2.866s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff91da43d57800.statahead_max=32 llite.lustre-ffff91da43d57800.statahead_max=32 1001 real 0m11.445s user 0m0.027s sys 0m1.422s 'ls -l' 1000 files with statahead: 11 sec statahead total: 128 statahead wrong: 2 agl total: 110 list_total: 31 fname_total: 77 hit_total: 4817 miss_total: 1360 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff91da43d57800.statahead_stats= statahead total: 128 statahead wrong: 2 agl total: 110 list_total: 31 fname_total: 77 hit_total: 4817 miss_total: 1360 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats= snapshot_time: 1790684337.856079697 secs.nsecs start_time: 1790684223.723048439 secs.nsecs elapsed_time: 114.133031258 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 630 (time 1790684350.77 total 10.00 last 63.00) - open/close 1202 (time 1790684360.77 total 20.00 last 57.20) - open/close 1736 (time 1790684370.77 total 30.00 last 53.40) - open/close 2199 (time 1790684380.77 total 40.00 last 46.29) - open/close 2723 (time 1790684390.78 total 50.01 last 52.36) - open/close 3294 (time 1790684400.78 total 60.02 last 57.07) - open/close 3914 (time 1790684410.79 total 70.02 last 61.97) - open/close 4552 (time 1790684420.79 total 80.03 last 63.78) - open/close 5147 (time 1790684430.81 total 90.04 last 59.40) - open/close 5720 (time 1790684440.83 total 100.06 last 57.19) - open/close 6227 (time 1790684450.84 total 110.07 last 50.64) - open/close 6668 (time 1790684460.88 total 120.11 last 43.92) - open/close 7225 (time 1790684470.89 total 130.13 last 55.63) - open/close 7802 (time 1790684480.90 total 140.13 last 57.68) - open/close 8342 (time 1790684490.91 total 150.14 last 53.95) - open/close 8872 (time 1790684500.91 total 160.15 last 52.96) - open/close 9440 (time 1790684510.92 total 170.15 last 56.77) total: 10000 open/close in 179.84 seconds: 55.60 ops/second llite.lustre-ffff91da43d57800.statahead_batch_max=0 llite.lustre-ffff91da43d57800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.804s user 0m0.328s sys 0m15.508s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91da43d57800.statahead_batch_max=32 llite.lustre-ffff91da43d57800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m57.449s user 0m0.321s sys 0m14.526s batched statahead RPC= hit count=9968 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats= snapshot_time: 1790684724.819992885 secs.nsecs start_time: 1790684597.740001204 secs.nsecs elapsed_time: 127.079991681 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=0 llite.lustre-ffff91da43d57800.statahead_max=2048 llite.lustre-ffff91da43d57800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=0 total: 100 open/close in 2.93 seconds: 34.18 ops/second llite.lustre-ffff91da43d57800.statahead_max=0 101 real 0m2.832s user 0m0.006s sys 0m0.351s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91da43d57800.statahead_max=2048 llite.lustre-ffff91da43d57800.statahead_max=2048 101 real 0m1.093s user 0m0.002s sys 0m0.164s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10067 miss_total: 33 swrong=0 ewrong=0 - open/close 624 (time 1790685095.19 total 10.02 last 62.30) total: 900 open/close in 14.83 seconds: 60.69 ops/second llite.lustre-ffff91da43d57800.statahead_max=0 1001 real 0m22.557s user 0m0.037s sys 0m2.875s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff91da43d57800.statahead_max=2048 llite.lustre-ffff91da43d57800.statahead_max=2048 1001 real 0m7.024s user 0m0.026s sys 0m1.240s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11066 miss_total: 34 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff91da43d57800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11066 miss_total: 34 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats= snapshot_time: 1790685164.020815521 secs.nsecs start_time: 1790684727.477178766 secs.nsecs elapsed_time: 436.543636755 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 656 (time 1790685176.44 total 10.00 last 65.58) - open/close 1330 (time 1790685186.45 total 20.01 last 67.35) - open/close 1988 (time 1790685196.45 total 30.01 last 65.78) - open/close 2645 (time 1790685206.46 total 40.02 last 65.65) - open/close 3277 (time 1790685216.49 total 50.06 last 62.98) - open/close 4019 (time 1790685226.51 total 60.07 last 74.10) - open/close 4686 (time 1790685236.53 total 70.09 last 66.54) - open/close 5357 (time 1790685246.53 total 80.10 last 67.07) - open/close 6094 (time 1790685256.54 total 90.10 last 73.66) - open/close 6700 (time 1790685266.55 total 100.11 last 60.55) - open/close 7321 (time 1790685276.56 total 110.12 last 62.05) - open/close 7992 (time 1790685286.56 total 120.13 last 67.06) - open/close 8644 (time 1790685296.57 total 130.14 last 65.13) - open/close 9281 (time 1790685306.58 total 140.14 last 63.68) - open/close 9916 (time 1790685316.60 total 150.16 last 63.37) total: 10000 open/close in 151.46 seconds: 66.02 ops/second llite.lustre-ffff91da43d57800.statahead_batch_max=0 llite.lustre-ffff91da43d57800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.852s user 0m0.390s sys 0m17.047s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91da43d57800.statahead_batch_max=256 llite.lustre-ffff91da43d57800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91da43d57800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.935s user 0m0.321s sys 0m15.060s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff91da43d57800.batch_stats= snapshot_time: 1790685445.207533552 secs.nsecs start_time: 1790685383.852522368 secs.nsecs elapsed_time: 61.355011184 secs.nsecs subreqs per batch batches %% cum %% 1: 94 41 41 2: 27 12 54 4: 19 8 62 8: 14 6 68 16: 11 4 73 32: 19 8 82 64: 4 1 83 128: 1 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff91da43d57800.statahead_batch_max=64 llite.lustre-ffff91da43d57800.statahead_max=128