== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:38:53 (1787852333) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=0 llite.lustre-ffff946776945000.statahead_max=32 llite.lustre-ffff946776945000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=0 total: 100 open/close in 3.81 seconds: 26.24 ops/second llite.lustre-ffff946776945000.statahead_max=0 101 real 0m2.662s user 0m0.003s sys 0m0.297s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff946776945000.statahead_max=32 llite.lustre-ffff946776945000.statahead_max=32 101 real 0m1.922s user 0m0.003s sys 0m0.183s 'ls -l' 100 files with statahead: 2 sec statahead total: 141 statahead wrong: 2 agl total: 120 list_total: 29 fname_total: 111 hit_total: 3889 miss_total: 1206 swrong=2 ewrong=2 - open/close 696 (time 1787852365.44 total 10.01 last 69.53) total: 900 open/close in 13.08 seconds: 68.82 ops/second llite.lustre-ffff946776945000.statahead_max=0 1001 real 0m22.639s user 0m0.031s sys 0m2.808s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff946776945000.statahead_max=32 llite.lustre-ffff946776945000.statahead_max=32 1001 real 0m11.075s user 0m0.027s sys 0m1.332s 'ls -l' 1000 files with statahead: 11 sec statahead total: 142 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 111 hit_total: 4887 miss_total: 1208 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff946776945000.statahead_stats= statahead total: 142 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 111 hit_total: 4887 miss_total: 1208 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats= snapshot_time: 1787852442.367685684 secs.nsecs start_time: 1787852339.238345732 secs.nsecs elapsed_time: 103.129339952 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 496 (time 1787852455.98 total 10.02 last 49.48) - open/close 1189 (time 1787852465.99 total 20.03 last 69.23) - open/close 1870 (time 1787852476.01 total 30.06 last 67.95) - open/close 2509 (time 1787852486.03 total 40.08 last 63.77) - open/close 3183 (time 1787852496.03 total 50.08 last 67.40) - open/close 3776 (time 1787852506.06 total 60.10 last 59.16) - open/close 4316 (time 1787852516.07 total 70.12 last 53.92) - open/close 4922 (time 1787852526.08 total 80.12 last 60.58) - open/close 5494 (time 1787852536.08 total 90.12 last 57.20) - open/close 6240 (time 1787852546.09 total 100.14 last 74.49) - open/close 7032 (time 1787852556.10 total 110.14 last 79.18) - open/close 7778 (time 1787852566.10 total 120.15 last 74.54) - open/close 8572 (time 1787852576.11 total 130.15 last 79.36) - open/close 9306 (time 1787852586.11 total 140.16 last 73.37) total: 10000 open/close in 149.78 seconds: 66.76 ops/second llite.lustre-ffff946776945000.statahead_batch_max=0 llite.lustre-ffff946776945000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.316s user 0m0.271s sys 0m14.743s unbatched statahead RPC=10004 hit count=9997 llite.lustre-ffff946776945000.statahead_batch_max=32 llite.lustre-ffff946776945000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.436s user 0m0.299s sys 0m13.373s batched statahead RPC= hit count=9969 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats= snapshot_time: 1787852797.059052846 secs.nsecs start_time: 1787852677.697149547 secs.nsecs elapsed_time: 119.361903299 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=0 llite.lustre-ffff946776945000.statahead_max=2048 llite.lustre-ffff946776945000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=0 total: 100 open/close in 2.89 seconds: 34.66 ops/second llite.lustre-ffff946776945000.statahead_max=0 101 real 0m2.331s user 0m0.005s sys 0m0.296s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff946776945000.statahead_max=2048 llite.lustre-ffff946776945000.statahead_max=2048 101 real 0m0.861s user 0m0.007s sys 0m0.196s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10068 miss_total: 32 swrong=0 ewrong=0 - open/close 813 (time 1787853123.26 total 10.01 last 81.22) total: 900 open/close in 11.30 seconds: 79.64 ops/second llite.lustre-ffff946776945000.statahead_max=0 1001 real 0m20.655s user 0m0.033s sys 0m2.815s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff946776945000.statahead_max=2048 llite.lustre-ffff946776945000.statahead_max=2048 1001 real 0m5.099s user 0m0.027s sys 0m1.212s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11067 miss_total: 33 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff946776945000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11067 miss_total: 33 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats= snapshot_time: 1787853182.776727722 secs.nsecs start_time: 1787852799.685348941 secs.nsecs elapsed_time: 383.091378781 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 788 (time 1787853195.45 total 10.02 last 78.68) - open/close 1431 (time 1787853205.46 total 20.03 last 64.22) - open/close 2211 (time 1787853215.48 total 30.04 last 77.91) - open/close 2871 (time 1787853225.49 total 40.05 last 65.91) - open/close 3557 (time 1787853235.50 total 50.06 last 68.56) - open/close 4199 (time 1787853245.50 total 60.07 last 64.16) - open/close 4988 (time 1787853255.50 total 70.07 last 78.90) - open/close 5881 (time 1787853265.51 total 80.07 last 89.25) - open/close 6649 (time 1787853275.52 total 90.08 last 76.71) - open/close 7393 (time 1787853285.52 total 100.08 last 74.39) - open/close 8107 (time 1787853295.52 total 110.09 last 71.40) - open/close 8782 (time 1787853305.53 total 120.09 last 67.46) - open/close 9463 (time 1787853315.54 total 130.11 last 67.99) total: 10000 open/close in 136.82 seconds: 73.09 ops/second llite.lustre-ffff946776945000.statahead_batch_max=0 llite.lustre-ffff946776945000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.703s user 0m0.280s sys 0m15.224s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff946776945000.statahead_batch_max=256 llite.lustre-ffff946776945000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff946776945000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.311s user 0m0.303s sys 0m14.723s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff946776945000.batch_stats= snapshot_time: 1787853439.670972150 secs.nsecs start_time: 1787853383.912258897 secs.nsecs elapsed_time: 55.758713253 secs.nsecs subreqs per batch batches %% cum %% 1: 103 42 42 2: 38 15 58 4: 25 10 68 8: 10 4 72 16: 11 4 77 32: 12 4 82 64: 7 2 85 128: 1 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff946776945000.statahead_batch_max=64 llite.lustre-ffff946776945000.statahead_max=128