== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:34:34 (1779874474) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=0 llite.lustre-ffff93ac837a3000.statahead_max=32 llite.lustre-ffff93ac837a3000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.51 ops/second llite.lustre-ffff93ac837a3000.statahead_max=0 101 real 0m3.270s user 0m0.006s sys 0m0.335s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff93ac837a3000.statahead_max=32 llite.lustre-ffff93ac837a3000.statahead_max=32 101 real 0m0.955s user 0m0.005s sys 0m0.168s 'ls -l' 100 files with statahead: 1 sec statahead total: 131 statahead wrong: 2 agl total: 125 list_total: 30 fname_total: 95 hit_total: 3546 miss_total: 1566 swrong=2 ewrong=2 - open/close 784 (time 1779874502.70 total 10.01 last 78.33) total: 900 open/close in 11.69 seconds: 77.00 ops/second llite.lustre-ffff93ac837a3000.statahead_max=0 1001 real 0m19.650s user 0m0.036s sys 0m2.938s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff93ac837a3000.statahead_max=32 llite.lustre-ffff93ac837a3000.statahead_max=32 1001 real 0m10.266s user 0m0.023s sys 0m1.471s 'ls -l' 1000 files with statahead: 10 sec statahead total: 132 statahead wrong: 2 agl total: 126 list_total: 31 fname_total: 95 hit_total: 4543 miss_total: 1569 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff93ac837a3000.statahead_stats= statahead total: 132 statahead wrong: 2 agl total: 126 list_total: 31 fname_total: 95 hit_total: 4543 miss_total: 1569 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats= snapshot_time: 1779874569.371643272 secs.nsecs start_time: 1779874479.016614592 secs.nsecs elapsed_time: 90.355028680 secs.nsecs subreqs per batch batches % cum % 1: 2 4 4 2: 1 2 7 4: 1 2 9 8: 2 4 14 16: 2 4 19 32: 34 80 100 - open/close 605 (time 1779874582.67 total 10.00 last 60.50) - open/close 1311 (time 1779874592.68 total 20.00 last 70.58) - open/close 2056 (time 1779874602.68 total 30.01 last 74.48) - open/close 2823 (time 1779874612.70 total 40.03 last 76.50) - open/close 3450 (time 1779874622.73 total 50.06 last 62.51) - open/close 4066 (time 1779874632.74 total 60.06 last 61.59) - open/close 4848 (time 1779874642.74 total 70.06 last 78.19) - open/close 5618 (time 1779874652.74 total 80.07 last 76.94) - open/close 6459 (time 1779874662.75 total 90.08 last 84.06) - open/close 7220 (time 1779874672.76 total 100.09 last 75.99) - open/close 7835 (time 1779874682.78 total 110.11 last 61.40) - open/close 8495 (time 1779874692.79 total 120.12 last 65.94) - open/close 9139 (time 1779874702.79 total 130.12 last 64.37) - open/close 9822 (time 1779874712.81 total 140.14 last 68.17) total: 10000 open/close in 143.62 seconds: 69.63 ops/second llite.lustre-ffff93ac837a3000.statahead_batch_max=0 llite.lustre-ffff93ac837a3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.111s user 0m0.374s sys 0m14.902s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93ac837a3000.statahead_batch_max=32 llite.lustre-ffff93ac837a3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.644s user 0m0.319s sys 0m14.242s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats= snapshot_time: 1779874926.152662175 secs.nsecs start_time: 1779874797.263976351 secs.nsecs elapsed_time: 128.888685824 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=0 llite.lustre-ffff93ac837a3000.statahead_max=2048 llite.lustre-ffff93ac837a3000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=0 total: 100 open/close in 2.18 seconds: 45.91 ops/second llite.lustre-ffff93ac837a3000.statahead_max=0 101 real 0m2.309s user 0m0.002s sys 0m0.333s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff93ac837a3000.statahead_max=2048 llite.lustre-ffff93ac837a3000.statahead_max=2048 101 real 0m0.800s user 0m0.001s sys 0m0.151s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 611 (time 1779875250.32 total 10.04 last 60.84) total: 900 open/close in 13.97 seconds: 64.45 ops/second llite.lustre-ffff93ac837a3000.statahead_max=0 1001 real 0m25.269s user 0m0.035s sys 0m3.040s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff93ac837a3000.statahead_max=2048 llite.lustre-ffff93ac837a3000.statahead_max=2048 1001 real 0m8.292s user 0m0.029s sys 0m1.409s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff93ac837a3000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats= snapshot_time: 1779875326.194713292 secs.nsecs start_time: 1779874928.539348999 secs.nsecs elapsed_time: 397.655364293 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 683 (time 1779875339.07 total 10.00 last 68.28) - open/close 1338 (time 1779875349.07 total 20.00 last 65.49) - open/close 2114 (time 1779875359.08 total 30.01 last 77.53) - open/close 2847 (time 1779875369.09 total 40.03 last 73.20) - open/close 3556 (time 1779875379.10 total 50.03 last 70.89) - open/close 4374 (time 1779875389.10 total 60.04 last 81.75) - open/close 4952 (time 1779875399.11 total 70.04 last 57.75) - open/close 5618 (time 1779875409.11 total 80.04 last 66.59) - open/close 6407 (time 1779875419.14 total 90.07 last 78.71) - open/close 7189 (time 1779875429.14 total 100.07 last 78.17) - open/close 8024 (time 1779875439.15 total 110.08 last 83.44) - open/close 8724 (time 1779875449.15 total 120.09 last 69.96) - open/close 9532 (time 1779875459.15 total 130.09 last 80.79) total: 10000 open/close in 137.85 seconds: 72.54 ops/second llite.lustre-ffff93ac837a3000.statahead_batch_max=0 llite.lustre-ffff93ac837a3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.024s user 0m0.313s sys 0m14.563s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93ac837a3000.statahead_batch_max=256 llite.lustre-ffff93ac837a3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93ac837a3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.221s user 0m0.321s sys 0m14.300s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93ac837a3000.batch_stats= snapshot_time: 1779875620.307698264 secs.nsecs start_time: 1779875546.001524415 secs.nsecs elapsed_time: 74.306173849 secs.nsecs subreqs per batch batches % cum % 1: 237 51 51 2: 69 15 66 4: 44 9 76 8: 35 7 83 16: 16 3 87 32: 25 5 92 64: 1 0 92 128: 0 0 92 256: 33 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93ac837a3000.statahead_batch_max=64 llite.lustre-ffff93ac837a3000.statahead_max=128