== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:18:31 (1789460311) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=0 llite.lustre-ffff932c111d9000.statahead_max=32 llite.lustre-ffff932c111d9000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.17 seconds: 46.10 ops/second llite.lustre-ffff932c111d9000.statahead_max=0 101 real 0m2.164s user 0m0.006s sys 0m0.320s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff932c111d9000.statahead_max=32 llite.lustre-ffff932c111d9000.statahead_max=32 101 real 0m0.976s user 0m0.003s sys 0m0.160s 'ls -l' 100 files with statahead: 1 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 30 fname_total: 105 hit_total: 3979 miss_total: 1467 swrong=2 ewrong=2 - open/close 860 (time 1789460338.48 total 10.01 last 85.93) total: 900 open/close in 10.57 seconds: 85.14 ops/second llite.lustre-ffff932c111d9000.statahead_max=0 1001 real 0m20.372s user 0m0.032s sys 0m2.997s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff932c111d9000.statahead_max=32 llite.lustre-ffff932c111d9000.statahead_max=32 1001 real 0m10.177s user 0m0.024s sys 0m1.371s 'ls -l' 1000 files with statahead: 10 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 31 fname_total: 105 hit_total: 4975 miss_total: 1471 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff932c111d9000.statahead_stats= statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 31 fname_total: 105 hit_total: 4975 miss_total: 1471 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789460405.808017367 secs.nsecs start_time: 1789460316.946518130 secs.nsecs elapsed_time: 88.861499237 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 3 7 10 8: 2 5 15 16: 18 46 61 32: 15 38 100 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789460405.809517733 secs.nsecs start_time: 1789460316.946737853 secs.nsecs elapsed_time: 88.862779880 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 5 5 8: 3 7 12 16: 23 58 71 32: 11 28 100 - open/close 724 (time 1789460419.18 total 10.00 last 72.39) - open/close 1551 (time 1789460429.18 total 20.01 last 82.66) - open/close 2498 (time 1789460439.19 total 30.01 last 94.64) - open/close 3355 (time 1789460449.19 total 40.01 last 85.69) - open/close 4168 (time 1789460459.20 total 50.02 last 81.24) - open/close 4935 (time 1789460469.20 total 60.02 last 76.68) - open/close 5848 (time 1789460479.20 total 70.02 last 91.30) - open/close 6709 (time 1789460489.21 total 80.04 last 85.98) - open/close 7662 (time 1789460499.22 total 90.04 last 95.25) - open/close 8518 (time 1789460509.23 total 100.06 last 85.47) - open/close 9441 (time 1789460519.24 total 110.06 last 92.23) total: 10000 open/close in 116.24 seconds: 86.03 ops/second llite.lustre-ffff932c111d9000.statahead_batch_max=0 llite.lustre-ffff932c111d9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.108s user 0m0.312s sys 0m14.960s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff932c111d9000.statahead_batch_max=32 llite.lustre-ffff932c111d9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.117s user 0m0.305s sys 0m13.756s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789460707.066925335 secs.nsecs start_time: 1789460596.844261822 secs.nsecs elapsed_time: 110.222663513 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789460707.066994808 secs.nsecs start_time: 1789460596.844408812 secs.nsecs elapsed_time: 110.222585996 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=0 llite.lustre-ffff932c111d9000.statahead_max=2048 llite.lustre-ffff932c111d9000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.70 seconds: 37.06 ops/second llite.lustre-ffff932c111d9000.statahead_max=0 101 real 0m1.990s user 0m0.004s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff932c111d9000.statahead_max=2048 llite.lustre-ffff932c111d9000.statahead_max=2048 101 real 0m0.924s user 0m0.004s sys 0m0.171s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 856 (time 1789460985.35 total 10.00 last 85.60) total: 900 open/close in 10.57 seconds: 85.11 ops/second llite.lustre-ffff932c111d9000.statahead_max=0 1001 real 0m20.167s user 0m0.029s sys 0m2.967s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff932c111d9000.statahead_max=2048 llite.lustre-ffff932c111d9000.statahead_max=2048 1001 real 0m6.646s user 0m0.031s sys 0m1.479s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff932c111d9000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789461045.031260006 secs.nsecs start_time: 1789460709.911941483 secs.nsecs elapsed_time: 335.119318523 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 28 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789461045.031326221 secs.nsecs start_time: 1789460709.912096719 secs.nsecs elapsed_time: 335.119229502 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 4 57 57 8: 0 0 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 911 (time 1789461057.85 total 10.01 last 91.02) - open/close 1699 (time 1789461067.85 total 20.01 last 78.78) - open/close 2579 (time 1789461077.87 total 30.02 last 87.89) - open/close 3496 (time 1789461087.88 total 40.04 last 91.60) - open/close 4442 (time 1789461097.88 total 50.04 last 94.56) - open/close 5328 (time 1789461107.89 total 60.05 last 88.52) - open/close 6263 (time 1789461117.90 total 70.06 last 93.41) - open/close 7336 (time 1789461127.90 total 80.06 last 107.28) - open/close 8269 (time 1789461137.91 total 90.07 last 93.22) - open/close 9218 (time 1789461147.92 total 100.07 last 94.84) total: 10000 open/close in 108.17 seconds: 92.45 ops/second llite.lustre-ffff932c111d9000.statahead_batch_max=0 llite.lustre-ffff932c111d9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.761s user 0m0.308s sys 0m14.785s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff932c111d9000.statahead_batch_max=256 llite.lustre-ffff932c111d9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff932c111d9000.stats=clear mdc.lustre-MDT0001-mdc-ffff932c111d9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.933s user 0m0.302s sys 0m15.803s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789461280.674797611 secs.nsecs start_time: 1789461218.781306994 secs.nsecs elapsed_time: 61.893490617 secs.nsecs subreqs per batch batches %% cum %% 1: 87 37 37 2: 29 12 50 4: 33 14 64 8: 18 7 72 16: 6 2 74 32: 14 6 80 64: 9 3 84 128: 1 0 85 256: 34 14 100 mdc.lustre-MDT0001-mdc-ffff932c111d9000.batch_stats= snapshot_time: 1789461280.674865651 secs.nsecs start_time: 1789461218.781368933 secs.nsecs elapsed_time: 61.893496718 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff932c111d9000.statahead_batch_max=64 llite.lustre-ffff932c111d9000.statahead_max=128