== sanity test 123ad: Verify batching statahead works correctly ========================================================== 00:08:11 (1786421291) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=0 llite.lustre-ffff88ac51f09800.statahead_max=32 llite.lustre-ffff88ac51f09800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=0 total: 100 open/close in 2.55 seconds: 39.16 ops/second llite.lustre-ffff88ac51f09800.statahead_max=0 101 real 0m3.244s user 0m0.001s sys 0m0.363s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff88ac51f09800.statahead_max=32 llite.lustre-ffff88ac51f09800.statahead_max=32 101 real 0m1.765s user 0m0.003s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 132 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 89 hit_total: 4035 miss_total: 1352 swrong=2 ewrong=2 - open/close 646 (time 1786421322.43 total 10.01 last 64.54) total: 900 open/close in 13.09 seconds: 68.75 ops/second llite.lustre-ffff88ac51f09800.statahead_max=0 1001 real 0m20.774s user 0m0.028s sys 0m3.147s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff88ac51f09800.statahead_max=32 llite.lustre-ffff88ac51f09800.statahead_max=32 1001 real 0m9.821s user 0m0.030s sys 0m1.518s 'ls -l' 1000 files with statahead: 10 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 31 fname_total: 89 hit_total: 5033 miss_total: 1354 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff88ac51f09800.statahead_stats= statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 31 fname_total: 89 hit_total: 5033 miss_total: 1354 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats= snapshot_time: 1786421391.981727418 secs.nsecs start_time: 1786421296.212728633 secs.nsecs elapsed_time: 95.768998785 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 505 (time 1786421405.50 total 10.01 last 50.44) - open/close 1056 (time 1786421415.50 total 20.02 last 55.08) - open/close 1676 (time 1786421425.51 total 30.02 last 61.97) - open/close 2289 (time 1786421435.51 total 40.02 last 61.30) - open/close 2958 (time 1786421445.52 total 50.04 last 66.80) - open/close 3508 (time 1786421455.54 total 60.06 last 54.88) - open/close 4016 (time 1786421465.54 total 70.06 last 50.79) - open/close 4630 (time 1786421475.55 total 80.07 last 61.37) - open/close 5209 (time 1786421485.56 total 90.07 last 57.86) - open/close 5787 (time 1786421495.58 total 100.10 last 57.65) - open/close 6508 (time 1786421505.59 total 110.10 last 72.06) - open/close 7209 (time 1786421515.59 total 120.11 last 70.08) - open/close 7930 (time 1786421525.60 total 130.12 last 72.01) - open/close 8754 (time 1786421535.61 total 140.13 last 82.31) - open/close 9489 (time 1786421545.62 total 150.13 last 73.46) total: 10000 open/close in 157.72 seconds: 63.40 ops/second llite.lustre-ffff88ac51f09800.statahead_batch_max=0 llite.lustre-ffff88ac51f09800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.569s user 0m0.364s sys 0m15.761s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88ac51f09800.statahead_batch_max=32 llite.lustre-ffff88ac51f09800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.527s user 0m0.308s sys 0m15.120s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats= snapshot_time: 1786421764.453882313 secs.nsecs start_time: 1786421637.000102017 secs.nsecs elapsed_time: 127.453780296 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=0 llite.lustre-ffff88ac51f09800.statahead_max=2048 llite.lustre-ffff88ac51f09800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=0 total: 100 open/close in 2.37 seconds: 42.22 ops/second llite.lustre-ffff88ac51f09800.statahead_max=0 101 real 0m2.225s user 0m0.004s sys 0m0.334s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff88ac51f09800.statahead_max=2048 llite.lustre-ffff88ac51f09800.statahead_max=2048 101 real 0m0.919s user 0m0.003s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 791 (time 1786422089.40 total 10.01 last 79.04) total: 900 open/close in 11.89 seconds: 75.72 ops/second llite.lustre-ffff88ac51f09800.statahead_max=0 1001 real 0m19.884s user 0m0.028s sys 0m3.149s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff88ac51f09800.statahead_max=2048 llite.lustre-ffff88ac51f09800.statahead_max=2048 1001 real 0m7.014s user 0m0.032s sys 0m1.436s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff88ac51f09800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats= snapshot_time: 1786422153.736209508 secs.nsecs start_time: 1786421766.928905776 secs.nsecs elapsed_time: 386.807303732 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 1 10 30 32: 2 20 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 778 (time 1786422166.74 total 10.00 last 77.77) - open/close 1467 (time 1786422176.75 total 20.01 last 68.86) - open/close 2166 (time 1786422186.76 total 30.02 last 69.83) - open/close 2962 (time 1786422196.76 total 40.03 last 79.57) - open/close 3654 (time 1786422206.79 total 50.06 last 68.99) - open/close 4567 (time 1786422216.79 total 60.06 last 91.29) - open/close 5175 (time 1786422226.81 total 70.07 last 60.69) - open/close 5904 (time 1786422236.82 total 80.08 last 72.83) - open/close 6779 (time 1786422246.82 total 90.09 last 87.48) - open/close 7494 (time 1786422256.83 total 100.09 last 71.44) - open/close 8309 (time 1786422266.84 total 110.11 last 81.42) - open/close 9041 (time 1786422276.85 total 120.11 last 73.13) - open/close 9682 (time 1786422286.86 total 130.12 last 64.04) total: 10000 open/close in 133.74 seconds: 74.77 ops/second llite.lustre-ffff88ac51f09800.statahead_batch_max=0 llite.lustre-ffff88ac51f09800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.318s user 0m0.324s sys 0m16.377s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88ac51f09800.statahead_batch_max=256 llite.lustre-ffff88ac51f09800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88ac51f09800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.899s user 0m0.310s sys 0m15.740s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88ac51f09800.batch_stats= snapshot_time: 1786422430.338213222 secs.nsecs start_time: 1786422359.524799810 secs.nsecs elapsed_time: 70.813413412 secs.nsecs subreqs per batch batches %% cum %% 1: 222 51 51 2: 67 15 67 4: 42 9 77 8: 29 6 84 16: 16 3 87 32: 15 3 91 64: 3 0 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88ac51f09800.statahead_batch_max=64 llite.lustre-ffff88ac51f09800.statahead_max=128