== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:18:05 (1789503485) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=0 llite.lustre-ffff98e56d83e000.statahead_max=32 llite.lustre-ffff98e56d83e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=0 total: 100 open/close in 2.76 seconds: 36.21 ops/second llite.lustre-ffff98e56d83e000.statahead_max=0 101 real 0m2.810s user 0m0.006s sys 0m0.348s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff98e56d83e000.statahead_max=32 llite.lustre-ffff98e56d83e000.statahead_max=32 101 real 0m1.288s user 0m0.002s sys 0m0.150s 'ls -l' 100 files with statahead: 2 sec statahead total: 124 statahead wrong: 2 agl total: 119 list_total: 30 fname_total: 94 hit_total: 4449 miss_total: 1278 swrong=2 ewrong=2 - open/close 595 (time 1789503516.79 total 10.02 last 59.40) total: 900 open/close in 15.24 seconds: 59.07 ops/second llite.lustre-ffff98e56d83e000.statahead_max=0 1001 real 0m26.700s user 0m0.036s sys 0m3.106s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff98e56d83e000.statahead_max=32 llite.lustre-ffff98e56d83e000.statahead_max=32 1001 real 0m10.995s user 0m0.028s sys 0m1.414s 'ls -l' 1000 files with statahead: 11 sec statahead total: 125 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 94 hit_total: 5444 miss_total: 1283 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff98e56d83e000.statahead_stats= statahead total: 125 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 94 hit_total: 5444 miss_total: 1283 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats= snapshot_time: 1789503597.156687904 secs.nsecs start_time: 1789503492.303448327 secs.nsecs elapsed_time: 104.853239577 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 519 (time 1789503609.71 total 10.05 last 51.63) - open/close 1140 (time 1789503619.72 total 20.06 last 62.03) - open/close 1651 (time 1789503629.73 total 30.07 last 51.06) - open/close 2170 (time 1789503639.74 total 40.08 last 51.85) - open/close 2715 (time 1789503649.75 total 50.09 last 54.45) - open/close 3185 (time 1789503659.76 total 60.10 last 46.97) - open/close 3656 (time 1789503669.76 total 70.10 last 47.09) - open/close 4244 (time 1789503679.77 total 80.11 last 58.72) - open/close 4888 (time 1789503689.77 total 90.11 last 64.40) - open/close 5509 (time 1789503699.79 total 100.12 last 62.03) - open/close 6136 (time 1789503709.80 total 110.14 last 62.63) - open/close 6748 (time 1789503719.80 total 120.14 last 61.18) - open/close 7313 (time 1789503729.83 total 130.17 last 56.34) - open/close 7924 (time 1789503739.84 total 140.18 last 61.06) - open/close 8512 (time 1789503749.85 total 150.19 last 58.70) - open/close 9076 (time 1789503759.86 total 160.20 last 56.38) - open/close 9652 (time 1789503769.87 total 170.21 last 57.54) total: 10000 open/close in 176.05 seconds: 56.80 ops/second llite.lustre-ffff98e56d83e000.statahead_batch_max=0 llite.lustre-ffff98e56d83e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.341s user 0m0.327s sys 0m14.475s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98e56d83e000.statahead_batch_max=32 llite.lustre-ffff98e56d83e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m56.200s user 0m0.312s sys 0m13.779s batched statahead RPC= hit count=9963 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats= snapshot_time: 1789503981.243327704 secs.nsecs start_time: 1789503852.000082605 secs.nsecs elapsed_time: 129.243245099 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=0 llite.lustre-ffff98e56d83e000.statahead_max=2048 llite.lustre-ffff98e56d83e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=0 total: 100 open/close in 2.95 seconds: 33.95 ops/second llite.lustre-ffff98e56d83e000.statahead_max=0 101 real 0m2.633s user 0m0.006s sys 0m0.342s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98e56d83e000.statahead_max=2048 llite.lustre-ffff98e56d83e000.statahead_max=2048 101 real 0m1.213s user 0m0.006s sys 0m0.159s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10062 miss_total: 38 swrong=0 ewrong=0 - open/close 590 (time 1789504358.31 total 10.00 last 58.99) total: 900 open/close in 15.04 seconds: 59.82 ops/second llite.lustre-ffff98e56d83e000.statahead_max=0 1001 real 0m25.194s user 0m0.033s sys 0m2.978s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff98e56d83e000.statahead_max=2048 llite.lustre-ffff98e56d83e000.statahead_max=2048 1001 real 0m7.236s user 0m0.034s sys 0m1.362s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11061 miss_total: 39 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff98e56d83e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11061 miss_total: 39 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats= snapshot_time: 1789504432.349067194 secs.nsecs start_time: 1789503984.444844567 secs.nsecs elapsed_time: 447.904222627 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 627 (time 1789504445.24 total 10.01 last 62.63) - open/close 1214 (time 1789504455.24 total 20.01 last 58.69) - open/close 1826 (time 1789504465.27 total 30.05 last 61.00) - open/close 2509 (time 1789504475.29 total 40.06 last 68.22) - open/close 3186 (time 1789504485.29 total 50.06 last 67.65) - open/close 3788 (time 1789504495.30 total 60.07 last 60.14) - open/close 4393 (time 1789504505.31 total 70.08 last 60.48) - open/close 5075 (time 1789504515.32 total 80.09 last 68.13) - open/close 5662 (time 1789504525.32 total 90.09 last 58.70) - open/close 6241 (time 1789504535.32 total 100.09 last 57.88) - open/close 6828 (time 1789504545.34 total 110.11 last 58.62) - open/close 7448 (time 1789504555.34 total 120.11 last 61.98) - open/close 8025 (time 1789504565.34 total 130.11 last 57.70) - open/close 8650 (time 1789504575.36 total 140.13 last 62.39) - open/close 9270 (time 1789504585.36 total 150.13 last 61.97) - open/close 9860 (time 1789504595.36 total 160.14 last 58.98) total: 10000 open/close in 162.49 seconds: 61.54 ops/second llite.lustre-ffff98e56d83e000.statahead_batch_max=0 llite.lustre-ffff98e56d83e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.427s user 0m0.335s sys 0m15.888s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98e56d83e000.statahead_batch_max=256 llite.lustre-ffff98e56d83e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98e56d83e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.776s user 0m0.309s sys 0m15.210s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98e56d83e000.batch_stats= snapshot_time: 1789504738.184907251 secs.nsecs start_time: 1789504671.118612543 secs.nsecs elapsed_time: 67.066294708 secs.nsecs subreqs per batch batches %% cum %% 1: 167 48 48 2: 45 13 61 4: 39 11 73 8: 22 6 79 16: 15 4 83 32: 14 4 88 64: 6 1 89 128: 1 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98e56d83e000.statahead_batch_max=64 llite.lustre-ffff98e56d83e000.statahead_max=128