== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:38:28 (1786376308) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=0 llite.lustre-ffff9c4e87a79000.statahead_max=32 llite.lustre-ffff9c4e87a79000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.00 seconds: 50.09 ops/second llite.lustre-ffff9c4e87a79000.statahead_max=0 101 real 0m2.059s user 0m0.002s sys 0m0.330s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c4e87a79000.statahead_max=32 llite.lustre-ffff9c4e87a79000.statahead_max=32 101 real 0m1.058s user 0m0.006s sys 0m0.214s 'ls -l' 100 files with statahead: 1 sec statahead total: 89 statahead wrong: 2 agl total: 85 list_total: 28 fname_total: 48 hit_total: 3566 miss_total: 1021 swrong=2 ewrong=2 total: 900 open/close in 9.13 seconds: 98.57 ops/second llite.lustre-ffff9c4e87a79000.statahead_max=0 1001 real 0m16.642s user 0m0.026s sys 0m3.070s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff9c4e87a79000.statahead_max=32 llite.lustre-ffff9c4e87a79000.statahead_max=32 1001 real 0m6.997s user 0m0.028s sys 0m1.459s 'ls -l' 1000 files with statahead: 7 sec statahead total: 90 statahead wrong: 2 agl total: 86 list_total: 29 fname_total: 48 hit_total: 4564 miss_total: 1023 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff9c4e87a79000.statahead_stats= statahead total: 90 statahead wrong: 2 agl total: 86 list_total: 29 fname_total: 48 hit_total: 4564 miss_total: 1023 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376382.452099812 secs.nsecs start_time: 1786376312.554683625 secs.nsecs elapsed_time: 69.897416187 secs.nsecs subreqs per batch batches %% cum %% 1: 3 8 8 2: 0 0 8 4: 3 8 16 8: 0 0 16 16: 26 70 86 32: 5 13 100 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376382.452354192 secs.nsecs start_time: 1786376312.554853188 secs.nsecs elapsed_time: 69.897501004 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 4 10 15 16: 8 20 35 32: 26 65 100 - open/close 834 (time 1786376394.56 total 10.01 last 83.28) - open/close 1769 (time 1786376404.57 total 20.03 last 93.40) - open/close 2743 (time 1786376414.60 total 30.05 last 97.17) - open/close 3747 (time 1786376424.61 total 40.06 last 100.27) - open/close 4784 (time 1786376434.61 total 50.07 last 103.66) - open/close 5684 (time 1786376444.62 total 60.08 last 89.91) - open/close 6677 (time 1786376454.63 total 70.08 last 99.23) - open/close 7768 (time 1786376464.63 total 80.08 last 109.10) - open/close 8703 (time 1786376474.63 total 90.09 last 93.47) - open/close 9739 (time 1786376484.64 total 100.09 last 103.59) total: 10000 open/close in 102.88 seconds: 97.20 ops/second llite.lustre-ffff9c4e87a79000.statahead_batch_max=0 llite.lustre-ffff9c4e87a79000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.348s user 0m0.306s sys 0m16.762s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c4e87a79000.statahead_batch_max=32 llite.lustre-ffff9c4e87a79000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m19.283s user 0m0.290s sys 0m15.367s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376633.743051667 secs.nsecs start_time: 1786376544.419203314 secs.nsecs elapsed_time: 89.323848353 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376633.745425219 secs.nsecs start_time: 1786376544.419429810 secs.nsecs elapsed_time: 89.325995409 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=0 llite.lustre-ffff9c4e87a79000.statahead_max=2048 llite.lustre-ffff9c4e87a79000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.09 seconds: 47.95 ops/second llite.lustre-ffff9c4e87a79000.statahead_max=0 101 real 0m1.822s user 0m0.007s sys 0m0.310s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c4e87a79000.statahead_max=2048 llite.lustre-ffff9c4e87a79000.statahead_max=2048 101 real 0m0.686s user 0m0.006s sys 0m0.178s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 789 (time 1786376883.04 total 10.00 last 78.86) total: 900 open/close in 11.29 seconds: 79.74 ops/second llite.lustre-ffff9c4e87a79000.statahead_max=0 1001 real 0m16.452s user 0m0.038s sys 0m2.975s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff9c4e87a79000.statahead_max=2048 llite.lustre-ffff9c4e87a79000.statahead_max=2048 1001 real 0m6.428s user 0m0.029s sys 0m1.663s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff9c4e87a79000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376930.415026221 secs.nsecs start_time: 1786376635.630952695 secs.nsecs elapsed_time: 294.784073526 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 3 37 50 8: 1 12 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786376930.415102328 secs.nsecs start_time: 1786376635.631969043 secs.nsecs elapsed_time: 294.783133285 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 2 25 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 1018 (time 1786376942.42 total 10.00 last 101.76) - open/close 2053 (time 1786376952.42 total 20.01 last 103.46) - open/close 2995 (time 1786376962.42 total 30.01 last 94.17) - open/close 3889 (time 1786376972.43 total 40.02 last 89.32) - open/close 4766 (time 1786376982.44 total 50.03 last 87.63) - open/close 5792 (time 1786376992.47 total 60.06 last 102.28) - open/close 6851 (time 1786377002.48 total 70.07 last 105.80) - open/close 7873 (time 1786377012.49 total 80.08 last 102.07) - open/close 8925 (time 1786377022.49 total 90.08 last 105.19) total: 10000 open/close in 99.80 seconds: 100.21 ops/second llite.lustre-ffff9c4e87a79000.statahead_batch_max=0 llite.lustre-ffff9c4e87a79000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.276s user 0m0.324s sys 0m17.378s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c4e87a79000.statahead_batch_max=256 llite.lustre-ffff9c4e87a79000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.stats=clear mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.199s user 0m0.316s sys 0m16.962s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786377144.304138546 secs.nsecs start_time: 1786377085.939387100 secs.nsecs elapsed_time: 58.364751446 secs.nsecs subreqs per batch batches %% cum %% 1: 241 55 55 2: 66 15 70 4: 44 10 80 8: 17 3 84 16: 14 3 87 32: 14 3 91 64: 4 0 91 128: 1 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff9c4e87a79000.batch_stats= snapshot_time: 1786377144.304216249 secs.nsecs start_time: 1786377085.939664948 secs.nsecs elapsed_time: 58.364551301 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9c4e87a79000.statahead_batch_max=64 llite.lustre-ffff9c4e87a79000.statahead_max=128