== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:51:26 (1785952286) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=0 llite.lustre-ffff8eb2332bc800.statahead_max=32 llite.lustre-ffff8eb2332bc800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=0 total: 100 open/close in 3.45 seconds: 28.99 ops/second llite.lustre-ffff8eb2332bc800.statahead_max=0 101 real 0m2.265s user 0m0.003s sys 0m0.318s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8eb2332bc800.statahead_max=32 llite.lustre-ffff8eb2332bc800.statahead_max=32 101 real 0m1.833s user 0m0.002s sys 0m0.158s 'ls -l' 100 files with statahead: 2 sec statahead total: 171 statahead wrong: 2 agl total: 164 list_total: 31 fname_total: 126 hit_total: 3786 miss_total: 1762 swrong=2 ewrong=2 - open/close 489 (time 1785952317.67 total 10.01 last 48.86) total: 900 open/close in 19.25 seconds: 46.76 ops/second llite.lustre-ffff8eb2332bc800.statahead_max=0 1001 real 0m37.485s user 0m0.028s sys 0m2.933s 'ls -l' 1000 files without statahead: 38 sec llite.lustre-ffff8eb2332bc800.statahead_max=32 llite.lustre-ffff8eb2332bc800.statahead_max=32 1001 real 0m19.968s user 0m0.032s sys 0m1.596s 'ls -l' 1000 files with statahead: 20 sec statahead total: 172 statahead wrong: 2 agl total: 165 list_total: 32 fname_total: 126 hit_total: 4782 miss_total: 1766 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff8eb2332bc800.statahead_stats= statahead total: 172 statahead wrong: 2 agl total: 165 list_total: 32 fname_total: 126 hit_total: 4782 miss_total: 1766 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats= snapshot_time: 1785952433.856455966 secs.nsecs start_time: 1785952291.707612776 secs.nsecs elapsed_time: 142.148843190 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 534 (time 1785952446.73 total 10.01 last 53.32) - open/close 1046 (time 1785952456.74 total 20.03 last 51.13) - open/close 1677 (time 1785952466.76 total 30.05 last 62.95) - open/close 2212 (time 1785952476.77 total 40.06 last 53.48) - open/close 2830 (time 1785952486.77 total 50.06 last 61.78) - open/close 3257 (time 1785952496.78 total 60.07 last 42.64) - open/close 3898 (time 1785952506.79 total 70.08 last 64.05) - open/close 4415 (time 1785952516.80 total 80.09 last 51.66) - open/close 5022 (time 1785952526.81 total 90.10 last 60.65) - open/close 5542 (time 1785952536.81 total 100.10 last 52.00) - open/close 6214 (time 1785952546.82 total 110.11 last 67.12) - open/close 6665 (time 1785952556.84 total 120.12 last 45.03) - open/close 7109 (time 1785952566.84 total 130.13 last 44.37) - open/close 7596 (time 1785952576.86 total 140.15 last 48.61) - open/close 8101 (time 1785952586.87 total 150.16 last 50.44) - open/close 8718 (time 1785952596.88 total 160.17 last 61.65) - open/close 9279 (time 1785952606.88 total 170.17 last 56.10) - open/close 9856 (time 1785952616.89 total 180.18 last 57.64) total: 10000 open/close in 182.40 seconds: 54.83 ops/second llite.lustre-ffff8eb2332bc800.statahead_batch_max=0 llite.lustre-ffff8eb2332bc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m29.427s user 0m0.287s sys 0m14.180s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8eb2332bc800.statahead_batch_max=32 llite.lustre-ffff8eb2332bc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m26.857s user 0m0.271s sys 0m13.849s batched statahead RPC= hit count=9965 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats= snapshot_time: 1785952887.346876434 secs.nsecs start_time: 1785952722.229651534 secs.nsecs elapsed_time: 165.117224900 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 1 32: 313 98 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=0 llite.lustre-ffff8eb2332bc800.statahead_max=2048 llite.lustre-ffff8eb2332bc800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=0 total: 100 open/close in 3.20 seconds: 31.28 ops/second llite.lustre-ffff8eb2332bc800.statahead_max=0 101 real 0m4.356s user 0m0.005s sys 0m0.327s 'ls -l' 100 files without statahead: 5 sec llite.lustre-ffff8eb2332bc800.statahead_max=2048 llite.lustre-ffff8eb2332bc800.statahead_max=2048 101 real 0m1.066s user 0m0.006s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10064 miss_total: 36 swrong=0 ewrong=0 - open/close 501 (time 1785953309.13 total 10.03 last 49.94) total: 900 open/close in 18.13 seconds: 49.64 ops/second llite.lustre-ffff8eb2332bc800.statahead_max=0 1001 real 0m26.920s user 0m0.030s sys 0m2.914s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8eb2332bc800.statahead_max=2048 llite.lustre-ffff8eb2332bc800.statahead_max=2048 1001 real 0m9.147s user 0m0.021s sys 0m1.271s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11063 miss_total: 37 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff8eb2332bc800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11063 miss_total: 37 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats= snapshot_time: 1785953401.871765227 secs.nsecs start_time: 1785952890.671165109 secs.nsecs elapsed_time: 511.200600118 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 896 (time 1785953414.98 total 10.00 last 89.57) - open/close 1355 (time 1785953425.01 total 20.03 last 45.77) - open/close 1823 (time 1785953435.02 total 30.04 last 46.75) - open/close 2560 (time 1785953445.02 total 40.04 last 73.68) - open/close 3182 (time 1785953455.03 total 50.05 last 62.15) - open/close 3691 (time 1785953465.05 total 60.07 last 50.81) - open/close 4119 (time 1785953475.05 total 70.07 last 42.78) - open/close 4527 (time 1785953485.09 total 80.11 last 40.64) - open/close 5002 (time 1785953495.11 total 90.13 last 47.40) - open/close 5531 (time 1785953505.12 total 100.14 last 52.86) - open/close 6091 (time 1785953515.12 total 110.15 last 55.98) - open/close 6816 (time 1785953525.12 total 120.15 last 72.50) - open/close 7431 (time 1785953535.13 total 130.15 last 61.49) - open/close 8033 (time 1785953545.13 total 140.15 last 60.16) - open/close 8559 (time 1785953555.14 total 150.16 last 52.56) - open/close 9059 (time 1785953565.15 total 160.18 last 49.92) - open/close 9504 (time 1785953575.17 total 170.19 last 44.44) total: 10000 open/close in 177.66 seconds: 56.29 ops/second llite.lustre-ffff8eb2332bc800.statahead_batch_max=0 llite.lustre-ffff8eb2332bc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m26.770s user 0m0.275s sys 0m13.970s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8eb2332bc800.statahead_batch_max=256 llite.lustre-ffff8eb2332bc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.076s user 0m0.245s sys 0m13.155s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8eb2332bc800.batch_stats= snapshot_time: 1785953779.216421271 secs.nsecs start_time: 1785953685.278120679 secs.nsecs elapsed_time: 93.938300592 secs.nsecs subreqs per batch batches %% cum %% 1: 172 43 43 2: 67 17 60 4: 55 13 74 8: 29 7 82 16: 14 3 85 32: 17 4 90 64: 5 1 91 128: 1 0 91 256: 33 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8eb2332bc800.statahead_batch_max=64 llite.lustre-ffff8eb2332bc800.statahead_max=128