== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:23:01 (1782224581) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=0 llite.lustre-ffff9b6a5006a800.statahead_max=32 llite.lustre-ffff9b6a5006a800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=0 total: 100 open/close in 3.56 seconds: 28.07 ops/second llite.lustre-ffff9b6a5006a800.statahead_max=0 101 real 0m3.210s user 0m0.008s sys 0m0.380s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b6a5006a800.statahead_max=32 llite.lustre-ffff9b6a5006a800.statahead_max=32 101 real 0m1.445s user 0m0.005s sys 0m0.220s 'ls -l' 100 files with statahead: 2 sec statahead total: 129 statahead wrong: 2 agl total: 124 list_total: 29 fname_total: 100 hit_total: 4264 miss_total: 1274 swrong=2 ewrong=2 - open/close 591 (time 1782224613.78 total 10.01 last 59.03) total: 900 open/close in 16.17 seconds: 55.67 ops/second llite.lustre-ffff9b6a5006a800.statahead_max=0 1001 real 0m26.015s user 0m0.040s sys 0m3.430s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9b6a5006a800.statahead_max=32 llite.lustre-ffff9b6a5006a800.statahead_max=32 1001 real 0m11.401s user 0m0.028s sys 0m1.685s 'ls -l' 1000 files with statahead: 11 sec statahead total: 130 statahead wrong: 2 agl total: 125 list_total: 30 fname_total: 100 hit_total: 5262 miss_total: 1276 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9b6a5006a800.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 125 list_total: 30 fname_total: 100 hit_total: 5262 miss_total: 1276 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats= snapshot_time: 1782224698.522945956 secs.nsecs start_time: 1782224587.950147308 secs.nsecs elapsed_time: 110.572798648 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 578 (time 1782224711.55 total 10.02 last 57.70) - open/close 1181 (time 1782224721.58 total 20.04 last 60.16) - open/close 1721 (time 1782224731.58 total 30.04 last 53.99) - open/close 2343 (time 1782224741.58 total 40.04 last 62.19) - open/close 2945 (time 1782224751.58 total 50.05 last 60.19) - open/close 3582 (time 1782224761.60 total 60.06 last 63.61) - open/close 4186 (time 1782224771.60 total 70.07 last 60.35) - open/close 4807 (time 1782224781.62 total 80.08 last 62.02) - open/close 5445 (time 1782224791.63 total 90.10 last 63.69) - open/close 6015 (time 1782224801.64 total 100.11 last 56.94) - open/close 6608 (time 1782224811.65 total 110.12 last 59.25) - open/close 7148 (time 1782224821.67 total 120.14 last 53.88) - open/close 7535 (time 1782224831.69 total 130.16 last 38.63) - open/close 8114 (time 1782224841.70 total 140.16 last 57.87) - open/close 8662 (time 1782224851.70 total 150.17 last 54.76) - open/close 9226 (time 1782224861.71 total 160.18 last 56.34) - open/close 9748 (time 1782224871.73 total 170.19 last 52.12) total: 10000 open/close in 176.18 seconds: 56.76 ops/second llite.lustre-ffff9b6a5006a800.statahead_batch_max=0 llite.lustre-ffff9b6a5006a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.021s user 0m0.368s sys 0m16.977s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b6a5006a800.statahead_batch_max=32 llite.lustre-ffff9b6a5006a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m0.521s user 0m0.353s sys 0m15.850s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats= snapshot_time: 1782225092.227156846 secs.nsecs start_time: 1782224955.740727374 secs.nsecs elapsed_time: 136.486429472 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=0 llite.lustre-ffff9b6a5006a800.statahead_max=2048 llite.lustre-ffff9b6a5006a800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=0 total: 100 open/close in 3.42 seconds: 29.21 ops/second llite.lustre-ffff9b6a5006a800.statahead_max=0 101 real 0m3.135s user 0m0.004s sys 0m0.399s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b6a5006a800.statahead_max=2048 llite.lustre-ffff9b6a5006a800.statahead_max=2048 101 real 0m1.104s user 0m0.007s sys 0m0.192s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 591 (time 1782225482.93 total 10.01 last 59.04) total: 900 open/close in 15.75 seconds: 57.14 ops/second llite.lustre-ffff9b6a5006a800.statahead_max=0 1001 real 0m26.557s user 0m0.051s sys 0m3.395s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9b6a5006a800.statahead_max=2048 llite.lustre-ffff9b6a5006a800.statahead_max=2048 1001 real 0m6.893s user 0m0.040s sys 0m1.824s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9b6a5006a800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats= snapshot_time: 1782225562.498567400 secs.nsecs start_time: 1782225094.832320731 secs.nsecs elapsed_time: 467.666246669 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 0 0 50 128: 3 25 75 256: 3 25 100 - open/close 648 (time 1782225575.93 total 10.01 last 64.77) - open/close 1212 (time 1782225585.94 total 20.02 last 56.34) - open/close 1795 (time 1782225595.95 total 30.03 last 58.25) - open/close 2376 (time 1782225605.97 total 40.04 last 58.01) - open/close 2964 (time 1782225615.97 total 50.04 last 58.80) - open/close 3567 (time 1782225625.97 total 60.04 last 60.28) - open/close 4170 (time 1782225635.98 total 70.05 last 60.27) - open/close 4721 (time 1782225645.99 total 80.06 last 55.05) - open/close 5282 (time 1782225656.00 total 90.07 last 56.03) - open/close 5823 (time 1782225666.00 total 100.07 last 54.09) - open/close 6411 (time 1782225676.01 total 110.08 last 58.76) - open/close 6997 (time 1782225686.02 total 120.10 last 58.51) - open/close 7533 (time 1782225696.04 total 130.11 last 53.51) - open/close 8099 (time 1782225706.05 total 140.12 last 56.55) - open/close 8581 (time 1782225716.07 total 150.14 last 48.11) - open/close 9084 (time 1782225726.07 total 160.14 last 50.28) - open/close 9612 (time 1782225736.08 total 170.15 last 52.77) total: 10000 open/close in 176.11 seconds: 56.78 ops/second llite.lustre-ffff9b6a5006a800.statahead_batch_max=0 llite.lustre-ffff9b6a5006a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.457s user 0m0.326s sys 0m16.754s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b6a5006a800.statahead_batch_max=256 llite.lustre-ffff9b6a5006a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.584s user 0m0.335s sys 0m17.234s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b6a5006a800.batch_stats= snapshot_time: 1782225881.124038211 secs.nsecs start_time: 1782225813.934818794 secs.nsecs elapsed_time: 67.189219417 secs.nsecs subreqs per batch batches %% cum %% 1: 264 56 56 2: 70 15 71 4: 42 9 80 8: 20 4 85 16: 20 4 89 32: 11 2 91 64: 4 0 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b6a5006a800.statahead_batch_max=64 llite.lustre-ffff9b6a5006a800.statahead_max=128