== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:20:01 (1786310401) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=0 llite.lustre-ffff9922bb5a6800.statahead_max=32 llite.lustre-ffff9922bb5a6800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.46 seconds: 40.58 ops/second llite.lustre-ffff9922bb5a6800.statahead_max=0 101 real 0m2.841s user 0m0.007s sys 0m0.363s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9922bb5a6800.statahead_max=32 llite.lustre-ffff9922bb5a6800.statahead_max=32 101 real 0m1.044s user 0m0.003s sys 0m0.176s 'ls -l' 100 files with statahead: 1 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 72 hit_total: 4107 miss_total: 1423 swrong=2 ewrong=2 - open/close 860 (time 1786310430.29 total 10.00 last 85.99) total: 900 open/close in 10.44 seconds: 86.24 ops/second llite.lustre-ffff9922bb5a6800.statahead_max=0 1001 real 0m18.407s user 0m0.030s sys 0m3.148s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff9922bb5a6800.statahead_max=32 llite.lustre-ffff9922bb5a6800.statahead_max=32 1001 real 0m8.341s user 0m0.030s sys 0m1.452s 'ls -l' 1000 files with statahead: 9 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 72 hit_total: 5106 miss_total: 1424 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff9922bb5a6800.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 32 fname_total: 72 hit_total: 5106 miss_total: 1424 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786310485.151796091 secs.nsecs start_time: 1786310407.054257188 secs.nsecs elapsed_time: 78.097538903 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 4 10 17 16: 16 41 58 32: 16 41 100 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786310485.151874881 secs.nsecs start_time: 1786310407.054348349 secs.nsecs elapsed_time: 78.097526532 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 1 2 7 4: 2 5 12 8: 1 2 15 16: 21 52 67 32: 13 32 100 - open/close 1023 (time 1786310496.94 total 10.00 last 102.27) - open/close 1961 (time 1786310506.94 total 20.01 last 93.76) - open/close 2749 (time 1786310516.97 total 30.03 last 78.59) - open/close 3684 (time 1786310526.97 total 40.03 last 93.50) - open/close 4652 (time 1786310536.97 total 50.04 last 96.79) - open/close 5668 (time 1786310546.98 total 60.04 last 101.51) - open/close 6554 (time 1786310556.99 total 70.05 last 88.52) - open/close 7536 (time 1786310566.99 total 80.05 last 98.19) - open/close 8587 (time 1786310576.99 total 90.06 last 105.05) - open/close 9440 (time 1786310587.00 total 100.07 last 85.22) total: 10000 open/close in 106.21 seconds: 94.15 ops/second llite.lustre-ffff9922bb5a6800.statahead_batch_max=0 llite.lustre-ffff9922bb5a6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.525s user 0m0.303s sys 0m16.360s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9922bb5a6800.statahead_batch_max=32 llite.lustre-ffff9922bb5a6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m29.039s user 0m0.304s sys 0m15.055s batched statahead RPC= hit count=9990 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786310764.893897300 secs.nsecs start_time: 1786310662.453545851 secs.nsecs elapsed_time: 102.440351449 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786310764.893969425 secs.nsecs start_time: 1786310662.453676087 secs.nsecs elapsed_time: 102.440293338 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=0 llite.lustre-ffff9922bb5a6800.statahead_max=2048 llite.lustre-ffff9922bb5a6800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.22 seconds: 45.10 ops/second llite.lustre-ffff9922bb5a6800.statahead_max=0 101 real 0m1.734s user 0m0.003s sys 0m0.341s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9922bb5a6800.statahead_max=2048 llite.lustre-ffff9922bb5a6800.statahead_max=2048 101 real 0m0.832s user 0m0.004s sys 0m0.210s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10089 miss_total: 11 swrong=0 ewrong=0 total: 900 open/close in 9.81 seconds: 91.75 ops/second llite.lustre-ffff9922bb5a6800.statahead_max=0 1001 real 0m18.018s user 0m0.031s sys 0m3.119s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff9922bb5a6800.statahead_max=2048 llite.lustre-ffff9922bb5a6800.statahead_max=2048 1001 real 0m7.756s user 0m0.024s sys 0m1.579s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11088 miss_total: 12 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9922bb5a6800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11088 miss_total: 12 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786311103.752056803 secs.nsecs start_time: 1786310767.165523894 secs.nsecs elapsed_time: 336.586532909 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 20 20 4: 1 10 30 8: 1 10 40 16: 2 20 60 32: 2 20 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786311103.752129033 secs.nsecs start_time: 1786310767.167435698 secs.nsecs elapsed_time: 336.584693335 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 10 10 8: 4 40 50 16: 1 10 60 32: 2 20 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 - open/close 857 (time 1786311116.13 total 10.00 last 85.66) - open/close 1749 (time 1786311126.13 total 20.01 last 89.16) - open/close 2599 (time 1786311136.14 total 30.02 last 84.92) - open/close 3470 (time 1786311146.15 total 40.03 last 87.02) - open/close 4427 (time 1786311156.16 total 50.03 last 95.67) - open/close 5440 (time 1786311166.16 total 60.04 last 101.23) - open/close 6443 (time 1786311176.18 total 70.05 last 100.17) - open/close 7444 (time 1786311186.18 total 80.06 last 100.04) - open/close 8266 (time 1786311196.19 total 90.06 last 82.16) - open/close 9005 (time 1786311206.20 total 100.07 last 73.81) - open/close 9731 (time 1786311216.21 total 110.08 last 72.55) total: 10000 open/close in 113.10 seconds: 88.42 ops/second llite.lustre-ffff9922bb5a6800.statahead_batch_max=0 llite.lustre-ffff9922bb5a6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.551s user 0m0.323s sys 0m16.619s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9922bb5a6800.statahead_batch_max=256 llite.lustre-ffff9922bb5a6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.stats=clear mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.487s user 0m0.317s sys 0m16.240s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786311368.129316061 secs.nsecs start_time: 1786311291.042800828 secs.nsecs elapsed_time: 77.086515233 secs.nsecs subreqs per batch batches %% cum %% 1: 183 48 48 2: 67 17 65 4: 33 8 74 8: 26 6 81 16: 13 3 84 32: 19 5 89 64: 4 1 91 128: 0 0 91 256: 34 8 100 mdc.lustre-MDT0001-mdc-ffff9922bb5a6800.batch_stats= snapshot_time: 1786311368.129383611 secs.nsecs start_time: 1786311291.045921026 secs.nsecs elapsed_time: 77.083462585 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9922bb5a6800.statahead_batch_max=64 llite.lustre-ffff9922bb5a6800.statahead_max=128