== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:00:38 (1789268438) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=0 llite.lustre-ffff8a134809b000.statahead_max=32 llite.lustre-ffff8a134809b000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.29 seconds: 43.70 ops/second llite.lustre-ffff8a134809b000.statahead_max=0 101 real 0m2.072s user 0m0.006s sys 0m0.291s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8a134809b000.statahead_max=32 llite.lustre-ffff8a134809b000.statahead_max=32 101 real 0m1.115s user 0m0.005s sys 0m0.181s 'ls -l' 100 files with statahead: 1 sec statahead total: 142 statahead wrong: 2 agl total: 136 list_total: 31 fname_total: 111 hit_total: 3711 miss_total: 1475 swrong=2 ewrong=2 - open/close 789 (time 1789268465.29 total 10.01 last 78.84) total: 900 open/close in 11.21 seconds: 80.25 ops/second llite.lustre-ffff8a134809b000.statahead_max=0 1001 real 0m20.104s user 0m0.030s sys 0m2.762s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8a134809b000.statahead_max=32 llite.lustre-ffff8a134809b000.statahead_max=32 1001 real 0m8.775s user 0m0.026s sys 0m1.359s 'ls -l' 1000 files with statahead: 9 sec statahead total: 143 statahead wrong: 2 agl total: 137 list_total: 32 fname_total: 111 hit_total: 4707 miss_total: 1479 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8a134809b000.statahead_stats= statahead total: 143 statahead wrong: 2 agl total: 137 list_total: 32 fname_total: 111 hit_total: 4707 miss_total: 1479 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789268528.914991870 secs.nsecs start_time: 1789268443.260453680 secs.nsecs elapsed_time: 85.654538190 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 3 7 10 16: 23 60 71 32: 11 28 100 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789268528.915057457 secs.nsecs start_time: 1789268443.260570803 secs.nsecs elapsed_time: 85.654486654 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 2 5 10 8: 2 5 15 16: 18 47 63 32: 14 36 100 - open/close 879 (time 1789268541.64 total 10.00 last 87.89) - open/close 1771 (time 1789268551.64 total 20.01 last 89.17) - open/close 2712 (time 1789268561.64 total 30.01 last 94.10) - open/close 3629 (time 1789268571.66 total 40.02 last 91.56) - open/close 4487 (time 1789268581.67 total 50.03 last 85.71) - open/close 5472 (time 1789268591.67 total 60.03 last 98.48) - open/close 6374 (time 1789268601.67 total 70.03 last 90.20) - open/close 7384 (time 1789268611.69 total 80.05 last 100.85) - open/close 8144 (time 1789268621.70 total 90.06 last 75.93) - open/close 8999 (time 1789268631.72 total 100.08 last 85.30) - open/close 9829 (time 1789268641.72 total 110.08 last 82.97) total: 10000 open/close in 112.03 seconds: 89.26 ops/second llite.lustre-ffff8a134809b000.statahead_batch_max=0 llite.lustre-ffff8a134809b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.184s user 0m0.312s sys 0m15.500s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a134809b000.statahead_batch_max=32 llite.lustre-ffff8a134809b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m34.809s user 0m0.292s sys 0m13.589s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789268811.075084466 secs.nsecs start_time: 1789268706.696837208 secs.nsecs elapsed_time: 104.378247258 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789268811.075389100 secs.nsecs start_time: 1789268706.696914573 secs.nsecs elapsed_time: 104.378474527 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=0 llite.lustre-ffff8a134809b000.statahead_max=2048 llite.lustre-ffff8a134809b000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.41 seconds: 41.45 ops/second llite.lustre-ffff8a134809b000.statahead_max=0 101 real 0m1.986s user 0m0.003s sys 0m0.298s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8a134809b000.statahead_max=2048 llite.lustre-ffff8a134809b000.statahead_max=2048 101 real 0m0.785s user 0m0.004s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 total: 900 open/close in 9.82 seconds: 91.67 ops/second llite.lustre-ffff8a134809b000.statahead_max=0 1001 real 0m18.148s user 0m0.034s sys 0m2.773s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff8a134809b000.statahead_max=2048 llite.lustre-ffff8a134809b000.statahead_max=2048 1001 real 0m4.825s user 0m0.024s sys 0m1.591s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff8a134809b000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789269150.543647013 secs.nsecs start_time: 1789268813.839815083 secs.nsecs elapsed_time: 336.703831930 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 3 37 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789269150.543720373 secs.nsecs start_time: 1789268813.839965030 secs.nsecs elapsed_time: 336.703755343 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 2 25 37 8: 1 12 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 869 (time 1789269162.58 total 10.01 last 86.85) - open/close 1692 (time 1789269172.59 total 20.01 last 82.23) - open/close 2613 (time 1789269182.60 total 30.02 last 92.02) - open/close 3503 (time 1789269192.61 total 40.03 last 88.92) - open/close 4443 (time 1789269202.61 total 50.04 last 93.93) - open/close 5393 (time 1789269212.62 total 60.05 last 94.92) - open/close 6330 (time 1789269222.63 total 70.05 last 93.68) - open/close 7323 (time 1789269232.63 total 80.06 last 99.23) - open/close 8130 (time 1789269242.64 total 90.06 last 80.66) - open/close 8872 (time 1789269252.64 total 100.07 last 74.15) - open/close 9706 (time 1789269262.65 total 110.07 last 83.37) total: 10000 open/close in 115.22 seconds: 86.79 ops/second llite.lustre-ffff8a134809b000.statahead_batch_max=0 llite.lustre-ffff8a134809b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.902s user 0m0.307s sys 0m17.872s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a134809b000.statahead_batch_max=256 llite.lustre-ffff8a134809b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a134809b000.stats=clear mdc.lustre-MDT0001-mdc-ffff8a134809b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.923s user 0m0.260s sys 0m15.402s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789269382.344642953 secs.nsecs start_time: 1789269320.869323261 secs.nsecs elapsed_time: 61.475319692 secs.nsecs subreqs per batch batches %% cum %% 1: 125 46 46 2: 30 11 57 4: 30 11 68 8: 20 7 75 16: 11 4 79 32: 11 4 83 64: 9 3 87 128: 1 0 87 256: 34 12 100 mdc.lustre-MDT0001-mdc-ffff8a134809b000.batch_stats= snapshot_time: 1789269382.344711362 secs.nsecs start_time: 1789269320.869640458 secs.nsecs elapsed_time: 61.475070904 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8a134809b000.statahead_batch_max=64 llite.lustre-ffff8a134809b000.statahead_max=128