== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:23:56 (1787037836) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=0 llite.lustre-ffff891fc54b7800.statahead_max=32 llite.lustre-ffff891fc54b7800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=0 total: 100 open/close in 2.04 seconds: 49.06 ops/second llite.lustre-ffff891fc54b7800.statahead_max=0 101 real 0m1.649s user 0m0.005s sys 0m0.315s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff891fc54b7800.statahead_max=32 llite.lustre-ffff891fc54b7800.statahead_max=32 101 real 0m0.924s user 0m0.004s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 30 fname_total: 97 hit_total: 4086 miss_total: 1345 swrong=2 ewrong=2 - open/close 806 (time 1787037860.43 total 10.00 last 80.57) total: 900 open/close in 11.13 seconds: 80.84 ops/second llite.lustre-ffff891fc54b7800.statahead_max=0 1001 real 0m18.227s user 0m0.040s sys 0m3.090s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff891fc54b7800.statahead_max=32 llite.lustre-ffff891fc54b7800.statahead_max=32 1001 real 0m10.678s user 0m0.025s sys 0m1.477s 'ls -l' 1000 files with statahead: 11 sec statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 31 fname_total: 97 hit_total: 5081 miss_total: 1350 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff891fc54b7800.statahead_stats= statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 31 fname_total: 97 hit_total: 5081 miss_total: 1350 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats= snapshot_time: 1787037922.138105340 secs.nsecs start_time: 1787037840.164934104 secs.nsecs elapsed_time: 81.973171236 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 623 (time 1787037934.97 total 10.01 last 62.24) - open/close 1093 (time 1787037944.99 total 20.03 last 46.93) - open/close 1948 (time 1787037955.00 total 30.04 last 85.42) - open/close 2797 (time 1787037965.00 total 40.04 last 84.83) - open/close 3687 (time 1787037975.01 total 50.05 last 88.97) - open/close 4429 (time 1787037985.01 total 60.05 last 74.17) - open/close 5129 (time 1787037995.03 total 70.07 last 69.88) - open/close 5785 (time 1787038005.04 total 80.08 last 65.53) - open/close 6500 (time 1787038015.05 total 90.09 last 71.45) - open/close 7386 (time 1787038025.06 total 100.10 last 88.51) - open/close 8133 (time 1787038035.06 total 110.10 last 74.65) - open/close 8933 (time 1787038045.08 total 120.12 last 79.89) - open/close 9748 (time 1787038055.09 total 130.13 last 81.36) total: 10000 open/close in 132.90 seconds: 75.24 ops/second llite.lustre-ffff891fc54b7800.statahead_batch_max=0 llite.lustre-ffff891fc54b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.833s user 0m0.305s sys 0m15.424s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff891fc54b7800.statahead_batch_max=32 llite.lustre-ffff891fc54b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m32.565s user 0m0.281s sys 0m14.747s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats= snapshot_time: 1787038229.013836834 secs.nsecs start_time: 1787038122.643096907 secs.nsecs elapsed_time: 106.370739927 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=0 llite.lustre-ffff891fc54b7800.statahead_max=2048 llite.lustre-ffff891fc54b7800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=0 total: 100 open/close in 2.16 seconds: 46.27 ops/second llite.lustre-ffff891fc54b7800.statahead_max=0 101 real 0m2.081s user 0m0.003s sys 0m0.333s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff891fc54b7800.statahead_max=2048 llite.lustre-ffff891fc54b7800.statahead_max=2048 101 real 0m0.955s user 0m0.005s sys 0m0.133s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 875 (time 1787038502.33 total 10.01 last 87.45) total: 900 open/close in 10.37 seconds: 86.82 ops/second llite.lustre-ffff891fc54b7800.statahead_max=0 1001 real 0m17.951s user 0m0.040s sys 0m3.064s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff891fc54b7800.statahead_max=2048 llite.lustre-ffff891fc54b7800.statahead_max=2048 1001 real 0m6.713s user 0m0.030s sys 0m1.406s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff891fc54b7800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats= snapshot_time: 1787038558.362729709 secs.nsecs start_time: 1787038232.315190824 secs.nsecs elapsed_time: 326.047538885 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 824 (time 1787038570.98 total 10.01 last 82.34) - open/close 1598 (time 1787038580.98 total 20.01 last 77.38) - open/close 2471 (time 1787038590.98 total 30.01 last 87.27) - open/close 3281 (time 1787038600.99 total 40.02 last 80.97) - open/close 4181 (time 1787038610.99 total 50.02 last 89.97) - open/close 5030 (time 1787038620.99 total 60.02 last 84.88) - open/close 5896 (time 1787038630.99 total 70.02 last 86.59) - open/close 6847 (time 1787038641.00 total 80.03 last 95.03) - open/close 7586 (time 1787038651.00 total 90.03 last 73.89) - open/close 8337 (time 1787038661.02 total 100.05 last 74.98) - open/close 9245 (time 1787038671.02 total 110.05 last 90.79) total: 10000 open/close in 118.60 seconds: 84.32 ops/second llite.lustre-ffff891fc54b7800.statahead_batch_max=0 llite.lustre-ffff891fc54b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.369s user 0m0.311s sys 0m16.404s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff891fc54b7800.statahead_batch_max=256 llite.lustre-ffff891fc54b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff891fc54b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.988s user 0m0.268s sys 0m15.821s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff891fc54b7800.batch_stats= snapshot_time: 1787038795.071120476 secs.nsecs start_time: 1787038738.864563331 secs.nsecs elapsed_time: 56.206557145 secs.nsecs subreqs per batch batches %% cum %% 1: 243 56 56 2: 59 13 70 4: 33 7 78 8: 28 6 84 16: 13 3 87 32: 14 3 90 64: 5 1 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff891fc54b7800.statahead_batch_max=64 llite.lustre-ffff891fc54b7800.statahead_max=128