== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:20:51 (1788913251) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=0 llite.lustre-ffff8a46f1fc5000.statahead_max=32 llite.lustre-ffff8a46f1fc5000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.69 seconds: 59.03 ops/second llite.lustre-ffff8a46f1fc5000.statahead_max=0 101 real 0m1.130s user 0m0.002s sys 0m0.280s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8a46f1fc5000.statahead_max=32 llite.lustre-ffff8a46f1fc5000.statahead_max=32 101 real 0m0.554s user 0m0.005s sys 0m0.155s 'ls -l' 100 files with statahead: 0 sec statahead total: 113 statahead wrong: 2 agl total: 108 list_total: 31 fname_total: 67 hit_total: 13750 miss_total: 1691 swrong=2 ewrong=2 total: 900 open/close in 7.16 seconds: 125.63 ops/second llite.lustre-ffff8a46f1fc5000.statahead_max=0 1001 real 0m16.049s user 0m0.039s sys 0m2.797s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff8a46f1fc5000.statahead_max=32 llite.lustre-ffff8a46f1fc5000.statahead_max=32 1001 real 0m10.101s user 0m0.026s sys 0m1.341s 'ls -l' 1000 files with statahead: 10 sec statahead total: 114 statahead wrong: 2 agl total: 109 list_total: 32 fname_total: 67 hit_total: 14747 miss_total: 1694 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8a46f1fc5000.statahead_stats= statahead total: 114 statahead wrong: 2 agl total: 109 list_total: 32 fname_total: 67 hit_total: 14747 miss_total: 1694 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913325.129534088 secs.nsecs start_time: 1788913253.375932618 secs.nsecs elapsed_time: 71.753601470 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 1 2 7 8: 4 10 17 16: 19 48 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913325.129592908 secs.nsecs start_time: 1788913253.375994177 secs.nsecs elapsed_time: 71.753598731 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 2 5 7 8: 2 5 12 16: 22 56 69 32: 12 30 100 - open/close 1001 (time 1788913338.10 total 10.00 last 100.10) - open/close 2053 (time 1788913348.10 total 20.00 last 105.16) - open/close 3179 (time 1788913358.10 total 30.01 last 112.58) - open/close 4356 (time 1788913368.11 total 40.01 last 117.60) - open/close 5576 (time 1788913378.13 total 50.03 last 121.83) - open/close 6629 (time 1788913388.13 total 60.03 last 105.29) - open/close 7661 (time 1788913398.14 total 70.04 last 103.12) - open/close 8714 (time 1788913408.14 total 80.04 last 105.23) - open/close 9737 (time 1788913418.15 total 90.05 last 102.23) total: 10000 open/close in 92.31 seconds: 108.33 ops/second llite.lustre-ffff8a46f1fc5000.statahead_batch_max=0 llite.lustre-ffff8a46f1fc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.590s user 0m0.263s sys 0m16.178s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a46f1fc5000.statahead_batch_max=32 llite.lustre-ffff8a46f1fc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.971s user 0m0.238s sys 0m13.263s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913538.976978642 secs.nsecs start_time: 1788913467.995118418 secs.nsecs elapsed_time: 70.981860224 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913538.977078142 secs.nsecs start_time: 1788913467.995197972 secs.nsecs elapsed_time: 70.981880170 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=0 llite.lustre-ffff8a46f1fc5000.statahead_max=2048 llite.lustre-ffff8a46f1fc5000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.93 seconds: 51.82 ops/second llite.lustre-ffff8a46f1fc5000.statahead_max=0 101 real 0m1.411s user 0m0.002s sys 0m0.284s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8a46f1fc5000.statahead_max=2048 llite.lustre-ffff8a46f1fc5000.statahead_max=2048 101 real 0m0.535s user 0m0.003s sys 0m0.152s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 total: 900 open/close in 7.62 seconds: 118.04 ops/second llite.lustre-ffff8a46f1fc5000.statahead_max=0 1001 real 0m15.712s user 0m0.025s sys 0m2.662s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff8a46f1fc5000.statahead_max=2048 llite.lustre-ffff8a46f1fc5000.statahead_max=2048 1001 real 0m3.741s user 0m0.027s sys 0m1.275s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 12 seconds llite.lustre-ffff8a46f1fc5000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913789.947898442 secs.nsecs start_time: 1788913540.457193691 secs.nsecs elapsed_time: 249.490704751 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 11 11 8: 4 44 55 16: 0 0 55 32: 1 11 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913789.947965804 secs.nsecs start_time: 1788913540.457355527 secs.nsecs elapsed_time: 249.490610277 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 33 33 8: 1 11 44 16: 2 22 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 - open/close 1209 (time 1788913801.94 total 10.00 last 120.86) - open/close 2368 (time 1788913811.95 total 20.01 last 115.83) - open/close 3333 (time 1788913821.96 total 30.02 last 96.41) - open/close 4448 (time 1788913831.97 total 40.03 last 111.38) - open/close 5649 (time 1788913841.97 total 50.04 last 120.04) - open/close 6828 (time 1788913851.98 total 60.04 last 117.81) - open/close 8008 (time 1788913861.99 total 70.05 last 117.92) - open/close 9183 (time 1788913871.99 total 80.05 last 117.49) total: 10000 open/close in 87.32 seconds: 114.53 ops/second llite.lustre-ffff8a46f1fc5000.statahead_batch_max=0 llite.lustre-ffff8a46f1fc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.421s user 0m0.279s sys 0m18.189s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a46f1fc5000.statahead_batch_max=256 llite.lustre-ffff8a46f1fc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m28.717s user 0m0.241s sys 0m15.518s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913953.703007923 secs.nsecs start_time: 1788913918.784522530 secs.nsecs elapsed_time: 34.918485393 secs.nsecs subreqs per batch batches %% cum %% 1: 132 53 53 2: 20 8 61 4: 26 10 71 8: 8 3 74 16: 6 2 77 32: 9 3 80 64: 12 4 85 128: 1 0 85 256: 35 14 100 mdc.lustre-MDT0001-mdc-ffff8a46f1fc5000.batch_stats= snapshot_time: 1788913953.703104201 secs.nsecs start_time: 1788913918.784650418 secs.nsecs elapsed_time: 34.918453783 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8a46f1fc5000.statahead_batch_max=64 llite.lustre-ffff8a46f1fc5000.statahead_max=128