== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:51:58 (1784191918) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=0 llite.lustre-ffff8de212251800.statahead_max=32 llite.lustre-ffff8de212251800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.59 seconds: 38.55 ops/second llite.lustre-ffff8de212251800.statahead_max=0 101 real 0m2.043s user 0m0.005s sys 0m0.334s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8de212251800.statahead_max=32 llite.lustre-ffff8de212251800.statahead_max=32 101 real 0m1.043s user 0m0.007s sys 0m0.194s 'ls -l' 100 files with statahead: 1 sec statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 31 fname_total: 88 hit_total: 3751 miss_total: 1702 swrong=2 ewrong=2 total: 900 open/close in 9.28 seconds: 97.01 ops/second llite.lustre-ffff8de212251800.statahead_max=0 1001 real 0m17.474s user 0m0.022s sys 0m3.097s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff8de212251800.statahead_max=32 llite.lustre-ffff8de212251800.statahead_max=32 1001 real 0m8.851s user 0m0.021s sys 0m1.396s 'ls -l' 1000 files with statahead: 9 sec statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 32 fname_total: 88 hit_total: 4750 miss_total: 1703 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8de212251800.statahead_stats= statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 32 fname_total: 88 hit_total: 4750 miss_total: 1703 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192001.970250089 secs.nsecs start_time: 1784191923.033154052 secs.nsecs elapsed_time: 78.937096037 secs.nsecs subreqs per batch batches %% cum %% 1: 3 8 8 2: 1 2 10 4: 1 2 13 8: 2 5 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192001.970320464 secs.nsecs start_time: 1784191923.037641768 secs.nsecs elapsed_time: 78.932678696 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 5 5 8: 2 5 10 16: 14 35 46 32: 21 53 100 - open/close 1063 (time 1784192014.09 total 10.01 last 106.22) - open/close 1978 (time 1784192024.10 total 20.02 last 91.42) - open/close 2986 (time 1784192034.10 total 30.02 last 100.75) - open/close 3992 (time 1784192044.11 total 40.03 last 100.52) - open/close 4715 (time 1784192054.11 total 50.03 last 72.27) - open/close 5481 (time 1784192064.12 total 60.04 last 76.55) - open/close 6220 (time 1784192074.12 total 70.04 last 73.89) - open/close 7113 (time 1784192084.12 total 80.04 last 89.27) - open/close 8032 (time 1784192094.14 total 90.06 last 91.77) - open/close 8891 (time 1784192104.14 total 100.06 last 85.89) - open/close 9613 (time 1784192114.14 total 110.06 last 72.19) total: 10000 open/close in 114.08 seconds: 87.66 ops/second llite.lustre-ffff8de212251800.statahead_batch_max=0 llite.lustre-ffff8de212251800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.736s user 0m0.291s sys 0m15.971s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8de212251800.statahead_batch_max=32 llite.lustre-ffff8de212251800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.481s user 0m0.324s sys 0m15.451s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192302.115230316 secs.nsecs start_time: 1784192187.974442220 secs.nsecs elapsed_time: 114.140788096 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192302.115299429 secs.nsecs start_time: 1784192187.974585428 secs.nsecs elapsed_time: 114.140714001 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=0 llite.lustre-ffff8de212251800.statahead_max=2048 llite.lustre-ffff8de212251800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.98 seconds: 50.54 ops/second llite.lustre-ffff8de212251800.statahead_max=0 101 real 0m1.973s user 0m0.006s sys 0m0.348s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8de212251800.statahead_max=2048 llite.lustre-ffff8de212251800.statahead_max=2048 101 real 0m0.857s user 0m0.005s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 869 (time 1784192587.87 total 10.00 last 86.87) total: 900 open/close in 10.43 seconds: 86.31 ops/second llite.lustre-ffff8de212251800.statahead_max=0 1001 real 0m18.255s user 0m0.029s sys 0m3.137s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff8de212251800.statahead_max=2048 llite.lustre-ffff8de212251800.statahead_max=2048 1001 real 0m6.253s user 0m0.028s sys 0m1.777s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff8de212251800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192642.651092472 secs.nsecs start_time: 1784192304.770332581 secs.nsecs elapsed_time: 337.880759891 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 11 11 8: 3 33 44 16: 2 22 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192642.651168584 secs.nsecs start_time: 1784192304.770469049 secs.nsecs elapsed_time: 337.880699535 secs.nsecs subreqs per batch batches %% cum %% 1: 1 11 11 2: 0 0 11 4: 2 22 33 8: 3 33 66 16: 0 0 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 - open/close 743 (time 1784192656.19 total 10.01 last 74.26) - open/close 1576 (time 1784192666.21 total 20.02 last 83.19) - open/close 2510 (time 1784192676.21 total 30.03 last 93.33) - open/close 3521 (time 1784192686.22 total 40.03 last 101.08) - open/close 4500 (time 1784192696.23 total 50.04 last 97.78) - open/close 5356 (time 1784192706.23 total 60.04 last 85.59) - open/close 6267 (time 1784192716.23 total 70.04 last 91.07) - open/close 7156 (time 1784192726.24 total 80.05 last 88.85) - open/close 7979 (time 1784192736.24 total 90.05 last 82.26) - open/close 8858 (time 1784192746.24 total 100.06 last 87.88) - open/close 9787 (time 1784192756.25 total 110.06 last 92.87) total: 10000 open/close in 112.35 seconds: 89.01 ops/second llite.lustre-ffff8de212251800.statahead_batch_max=0 llite.lustre-ffff8de212251800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.257s user 0m0.321s sys 0m16.922s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8de212251800.statahead_batch_max=256 llite.lustre-ffff8de212251800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8de212251800.stats=clear mdc.lustre-MDT0001-mdc-ffff8de212251800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.046s user 0m0.302s sys 0m15.247s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192901.590036296 secs.nsecs start_time: 1784192829.889220157 secs.nsecs elapsed_time: 71.700816139 secs.nsecs subreqs per batch batches %% cum %% 1: 223 51 51 2: 75 17 68 4: 41 9 78 8: 23 5 83 16: 15 3 87 32: 17 3 91 64: 4 0 92 128: 0 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff8de212251800.batch_stats= snapshot_time: 1784192901.590105272 secs.nsecs start_time: 1784192829.889429539 secs.nsecs elapsed_time: 71.700675733 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8de212251800.statahead_batch_max=64 llite.lustre-ffff8de212251800.statahead_max=128