== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:01:57 (1786752117) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=0 llite.lustre-ffff9d50c86f0800.statahead_max=32 llite.lustre-ffff9d50c86f0800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=0 total: 100 open/close in 3.31 seconds: 30.23 ops/second llite.lustre-ffff9d50c86f0800.statahead_max=0 101 real 0m2.601s user 0m0.006s sys 0m0.336s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9d50c86f0800.statahead_max=32 llite.lustre-ffff9d50c86f0800.statahead_max=32 101 real 0m1.044s user 0m0.006s sys 0m0.175s 'ls -l' 100 files with statahead: 1 sec statahead total: 125 statahead wrong: 2 agl total: 119 list_total: 30 fname_total: 90 hit_total: 3823 miss_total: 1534 swrong=2 ewrong=2 - open/close 675 (time 1786752146.29 total 10.00 last 67.49) total: 900 open/close in 13.28 seconds: 67.75 ops/second llite.lustre-ffff9d50c86f0800.statahead_max=0 1001 real 0m21.888s user 0m0.033s sys 0m3.051s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9d50c86f0800.statahead_max=32 llite.lustre-ffff9d50c86f0800.statahead_max=32 1001 real 0m8.996s user 0m0.027s sys 0m1.635s 'ls -l' 1000 files with statahead: 9 sec statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 90 hit_total: 4821 miss_total: 1536 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9d50c86f0800.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 90 hit_total: 4821 miss_total: 1536 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats= snapshot_time: 1786752210.635498375 secs.nsecs start_time: 1786752122.633819474 secs.nsecs elapsed_time: 88.001678901 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 815 (time 1786752223.05 total 10.01 last 81.42) - open/close 1606 (time 1786752233.05 total 20.01 last 79.07) - open/close 2395 (time 1786752243.06 total 30.02 last 78.82) - open/close 3205 (time 1786752253.08 total 40.04 last 80.86) - open/close 3995 (time 1786752263.09 total 50.05 last 78.93) - open/close 4777 (time 1786752273.10 total 60.06 last 78.13) - open/close 5455 (time 1786752283.10 total 70.06 last 67.77) - open/close 6178 (time 1786752293.11 total 80.07 last 72.25) - open/close 6927 (time 1786752303.12 total 90.08 last 74.86) - open/close 7662 (time 1786752313.12 total 100.08 last 73.46) - open/close 8405 (time 1786752323.12 total 110.08 last 74.30) - open/close 9191 (time 1786752333.13 total 120.09 last 78.54) - open/close 9901 (time 1786752343.14 total 130.10 last 70.95) total: 10000 open/close in 131.31 seconds: 76.16 ops/second llite.lustre-ffff9d50c86f0800.statahead_batch_max=0 llite.lustre-ffff9d50c86f0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.563s user 0m0.356s sys 0m17.816s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d50c86f0800.statahead_batch_max=32 llite.lustre-ffff9d50c86f0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.047s user 0m0.292s sys 0m15.422s batched statahead RPC= hit count=9991 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats= snapshot_time: 1786752544.532434844 secs.nsecs start_time: 1786752425.089363433 secs.nsecs elapsed_time: 119.443071411 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=0 llite.lustre-ffff9d50c86f0800.statahead_max=2048 llite.lustre-ffff9d50c86f0800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=0 total: 100 open/close in 2.47 seconds: 40.47 ops/second llite.lustre-ffff9d50c86f0800.statahead_max=0 101 real 0m2.953s user 0m0.008s sys 0m0.337s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9d50c86f0800.statahead_max=2048 llite.lustre-ffff9d50c86f0800.statahead_max=2048 101 real 0m1.145s user 0m0.004s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10090 miss_total: 10 swrong=0 ewrong=0 - open/close 744 (time 1786752861.96 total 10.02 last 74.21) total: 900 open/close in 12.16 seconds: 74.00 ops/second llite.lustre-ffff9d50c86f0800.statahead_max=0 1001 real 0m19.935s user 0m0.032s sys 0m2.983s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9d50c86f0800.statahead_max=2048 llite.lustre-ffff9d50c86f0800.statahead_max=2048 1001 real 0m6.301s user 0m0.032s sys 0m1.632s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11089 miss_total: 11 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff9d50c86f0800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11089 miss_total: 11 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats= snapshot_time: 1786752916.451951604 secs.nsecs start_time: 1786752547.733187290 secs.nsecs elapsed_time: 368.718764314 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 894 (time 1786752928.31 total 10.01 last 89.29) - open/close 1728 (time 1786752938.32 total 20.02 last 83.34) - open/close 2576 (time 1786752948.32 total 30.02 last 84.78) - open/close 3402 (time 1786752958.33 total 40.03 last 82.55) - open/close 4117 (time 1786752968.34 total 50.04 last 71.42) - open/close 4815 (time 1786752978.35 total 60.05 last 69.72) - open/close 5560 (time 1786752988.36 total 70.06 last 74.43) - open/close 6287 (time 1786752998.36 total 80.06 last 72.67) - open/close 7029 (time 1786753008.36 total 90.07 last 74.18) - open/close 7781 (time 1786753018.37 total 100.07 last 75.14) - open/close 8611 (time 1786753028.38 total 110.08 last 82.93) - open/close 9353 (time 1786753038.38 total 120.09 last 74.17) total: 10000 open/close in 128.67 seconds: 77.72 ops/second llite.lustre-ffff9d50c86f0800.statahead_batch_max=0 llite.lustre-ffff9d50c86f0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.468s user 0m0.329s sys 0m17.366s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d50c86f0800.statahead_batch_max=256 llite.lustre-ffff9d50c86f0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.678s user 0m0.271s sys 0m17.374s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d50c86f0800.batch_stats= snapshot_time: 1786753180.773377186 secs.nsecs start_time: 1786753114.916141189 secs.nsecs elapsed_time: 65.857235997 secs.nsecs subreqs per batch batches %% cum %% 1: 297 57 57 2: 82 15 72 4: 50 9 82 8: 27 5 87 16: 14 2 90 32: 10 1 92 64: 6 1 93 128: 0 0 93 256: 34 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d50c86f0800.statahead_batch_max=64 llite.lustre-ffff9d50c86f0800.statahead_max=128