== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:22:49 (1786749769) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=0 llite.lustre-ffff8bacad452800.statahead_max=32 llite.lustre-ffff8bacad452800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=0 total: 100 open/close in 3.11 seconds: 32.18 ops/second llite.lustre-ffff8bacad452800.statahead_max=0 101 real 0m2.572s user 0m0.006s sys 0m0.345s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8bacad452800.statahead_max=32 llite.lustre-ffff8bacad452800.statahead_max=32 101 real 0m1.214s user 0m0.007s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 116 statahead wrong: 2 agl total: 110 list_total: 30 fname_total: 70 hit_total: 3847 miss_total: 1611 swrong=2 ewrong=2 - open/close 739 (time 1786749798.98 total 10.01 last 73.83) total: 900 open/close in 12.43 seconds: 72.41 ops/second llite.lustre-ffff8bacad452800.statahead_max=0 1001 real 0m23.682s user 0m0.033s sys 0m3.192s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8bacad452800.statahead_max=32 llite.lustre-ffff8bacad452800.statahead_max=32 1001 real 0m11.145s user 0m0.037s sys 0m1.702s 'ls -l' 1000 files with statahead: 11 sec statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 70 hit_total: 4845 miss_total: 1613 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8bacad452800.statahead_stats= statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 70 hit_total: 4845 miss_total: 1613 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats= snapshot_time: 1786749875.778767610 secs.nsecs start_time: 1786749774.636166524 secs.nsecs elapsed_time: 101.142601086 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 665 (time 1786749888.41 total 10.00 last 66.48) - open/close 1320 (time 1786749898.41 total 20.00 last 65.49) - open/close 2067 (time 1786749908.43 total 30.02 last 74.61) - open/close 2821 (time 1786749918.43 total 40.02 last 75.39) - open/close 3532 (time 1786749928.43 total 50.02 last 71.07) - open/close 4220 (time 1786749938.44 total 60.03 last 68.76) - open/close 4877 (time 1786749948.46 total 70.05 last 65.54) - open/close 5572 (time 1786749958.47 total 80.06 last 69.42) - open/close 6269 (time 1786749968.49 total 90.08 last 69.58) - open/close 6793 (time 1786749978.49 total 100.09 last 52.38) - open/close 7394 (time 1786749988.50 total 110.09 last 60.08) - open/close 8007 (time 1786749998.50 total 120.09 last 61.26) - open/close 8687 (time 1786750008.51 total 130.10 last 67.93) - open/close 9338 (time 1786750018.54 total 140.13 last 64.92) - open/close 9991 (time 1786750028.55 total 150.14 last 65.25) total: 10000 open/close in 150.33 seconds: 66.52 ops/second llite.lustre-ffff8bacad452800.statahead_batch_max=0 llite.lustre-ffff8bacad452800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.805s user 0m0.328s sys 0m17.354s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8bacad452800.statahead_batch_max=32 llite.lustre-ffff8bacad452800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.008s user 0m0.340s sys 0m16.401s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats= snapshot_time: 1786750228.661332924 secs.nsecs start_time: 1786750106.896066901 secs.nsecs elapsed_time: 121.765266023 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=0 llite.lustre-ffff8bacad452800.statahead_max=2048 llite.lustre-ffff8bacad452800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=0 total: 100 open/close in 3.26 seconds: 30.71 ops/second llite.lustre-ffff8bacad452800.statahead_max=0 101 real 0m2.766s user 0m0.005s sys 0m0.326s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8bacad452800.statahead_max=2048 llite.lustre-ffff8bacad452800.statahead_max=2048 101 real 0m1.151s user 0m0.007s sys 0m0.195s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 648 (time 1786750578.05 total 10.02 last 64.70) total: 900 open/close in 13.59 seconds: 66.21 ops/second llite.lustre-ffff8bacad452800.statahead_max=0 1001 real 0m26.913s user 0m0.029s sys 0m3.263s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8bacad452800.statahead_max=2048 llite.lustre-ffff8bacad452800.statahead_max=2048 1001 real 0m9.741s user 0m0.027s sys 0m1.587s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8bacad452800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats= snapshot_time: 1786750656.427367499 secs.nsecs start_time: 1786750231.048418144 secs.nsecs elapsed_time: 425.378949355 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 734 (time 1786750669.20 total 10.00 last 73.38) - open/close 1397 (time 1786750679.21 total 20.01 last 66.26) - open/close 2116 (time 1786750689.22 total 30.02 last 71.82) - open/close 2809 (time 1786750699.23 total 40.03 last 69.24) - open/close 3520 (time 1786750709.23 total 50.03 last 71.08) - open/close 4148 (time 1786750719.27 total 60.07 last 62.56) - open/close 4705 (time 1786750729.29 total 70.10 last 55.56) - open/close 5291 (time 1786750739.29 total 80.10 last 58.58) - open/close 6047 (time 1786750749.30 total 90.10 last 75.59) - open/close 6752 (time 1786750759.30 total 100.11 last 70.45) - open/close 7385 (time 1786750769.31 total 110.11 last 63.29) - open/close 8006 (time 1786750779.31 total 120.11 last 62.07) - open/close 8606 (time 1786750789.32 total 130.12 last 59.93) - open/close 9358 (time 1786750799.32 total 140.13 last 75.18) total: 10000 open/close in 148.50 seconds: 67.34 ops/second llite.lustre-ffff8bacad452800.statahead_batch_max=0 llite.lustre-ffff8bacad452800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.859s user 0m0.328s sys 0m17.937s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8bacad452800.statahead_batch_max=256 llite.lustre-ffff8bacad452800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8bacad452800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.267s user 0m0.282s sys 0m17.162s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8bacad452800.batch_stats= snapshot_time: 1786750952.179204267 secs.nsecs start_time: 1786750876.656174222 secs.nsecs elapsed_time: 75.523030045 secs.nsecs subreqs per batch batches %% cum %% 1: 268 53 53 2: 90 17 71 4: 44 8 80 8: 33 6 86 16: 18 3 90 32: 11 2 92 64: 3 0 93 128: 0 0 93 256: 34 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8bacad452800.statahead_batch_max=64 llite.lustre-ffff8bacad452800.statahead_max=128