== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:04:01 (1784941441) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=0 llite.lustre-ffff9676c3342800.statahead_max=32 llite.lustre-ffff9676c3342800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=0 total: 100 open/close in 2.85 seconds: 35.05 ops/second llite.lustre-ffff9676c3342800.statahead_max=0 101 real 0m2.601s user 0m0.002s sys 0m0.342s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9676c3342800.statahead_max=32 llite.lustre-ffff9676c3342800.statahead_max=32 101 real 0m1.433s user 0m0.008s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 30 fname_total: 97 hit_total: 3650 miss_total: 1294 swrong=2 ewrong=2 - open/close 707 (time 1784941470.63 total 10.01 last 70.60) total: 900 open/close in 12.92 seconds: 69.63 ops/second llite.lustre-ffff9676c3342800.statahead_max=0 1001 real 0m21.919s user 0m0.032s sys 0m3.471s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9676c3342800.statahead_max=32 llite.lustre-ffff9676c3342800.statahead_max=32 1001 real 0m8.814s user 0m0.021s sys 0m1.680s 'ls -l' 1000 files with statahead: 9 sec statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 31 fname_total: 97 hit_total: 4648 miss_total: 1296 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9676c3342800.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 31 fname_total: 97 hit_total: 4648 miss_total: 1296 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats= snapshot_time: 1784941533.530573473 secs.nsecs start_time: 1784941446.947202969 secs.nsecs elapsed_time: 86.583370504 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 748 (time 1784941545.90 total 10.01 last 74.76) - open/close 1491 (time 1784941555.90 total 20.01 last 74.28) - open/close 2181 (time 1784941565.91 total 30.02 last 68.94) - open/close 2937 (time 1784941575.92 total 40.02 last 75.55) - open/close 3657 (time 1784941585.93 total 50.03 last 71.93) - open/close 4372 (time 1784941595.94 total 60.04 last 71.44) - open/close 5049 (time 1784941605.95 total 70.06 last 67.60) - open/close 5673 (time 1784941615.96 total 80.06 last 62.37) - open/close 6296 (time 1784941625.96 total 90.07 last 62.26) - open/close 6978 (time 1784941635.97 total 100.07 last 68.18) - open/close 7700 (time 1784941645.97 total 110.08 last 72.15) - open/close 8373 (time 1784941655.99 total 120.09 last 67.19) - open/close 8992 (time 1784941666.00 total 130.11 last 61.83) - open/close 9631 (time 1784941676.02 total 140.13 last 63.78) total: 10000 open/close in 145.79 seconds: 68.59 ops/second llite.lustre-ffff9676c3342800.statahead_batch_max=0 llite.lustre-ffff9676c3342800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.628s user 0m0.317s sys 0m18.727s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9676c3342800.statahead_batch_max=32 llite.lustre-ffff9676c3342800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.804s user 0m0.301s sys 0m16.518s batched statahead RPC= hit count=9989 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats= snapshot_time: 1784941845.061379661 secs.nsecs start_time: 1784941745.842140577 secs.nsecs elapsed_time: 99.219239084 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=0 llite.lustre-ffff9676c3342800.statahead_max=2048 llite.lustre-ffff9676c3342800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=0 total: 100 open/close in 2.63 seconds: 37.98 ops/second llite.lustre-ffff9676c3342800.statahead_max=0 101 real 0m2.362s user 0m0.009s sys 0m0.382s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9676c3342800.statahead_max=2048 llite.lustre-ffff9676c3342800.statahead_max=2048 101 real 0m0.785s user 0m0.007s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10088 miss_total: 12 swrong=0 ewrong=0 - open/close 776 (time 1784942143.18 total 10.01 last 77.53) total: 900 open/close in 11.71 seconds: 76.83 ops/second llite.lustre-ffff9676c3342800.statahead_max=0 1001 real 0m20.470s user 0m0.033s sys 0m3.147s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9676c3342800.statahead_max=2048 llite.lustre-ffff9676c3342800.statahead_max=2048 1001 real 0m5.565s user 0m0.026s sys 0m1.772s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9676c3342800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats= snapshot_time: 1784942198.555796996 secs.nsecs start_time: 1784941847.445138234 secs.nsecs elapsed_time: 351.110658762 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 704 (time 1784942210.43 total 10.00 last 70.37) - open/close 1435 (time 1784942220.44 total 20.01 last 73.08) - open/close 2142 (time 1784942230.44 total 30.01 last 70.69) - open/close 2864 (time 1784942240.44 total 40.01 last 72.18) - open/close 3621 (time 1784942250.45 total 50.02 last 75.65) - open/close 4412 (time 1784942260.45 total 60.02 last 79.06) - open/close 5219 (time 1784942270.45 total 70.02 last 80.69) - open/close 6009 (time 1784942280.46 total 80.03 last 78.94) - open/close 6739 (time 1784942290.47 total 90.04 last 72.96) - open/close 7477 (time 1784942300.47 total 100.04 last 73.77) - open/close 8182 (time 1784942310.49 total 110.06 last 70.36) - open/close 8931 (time 1784942320.49 total 120.06 last 74.88) - open/close 9705 (time 1784942330.50 total 130.07 last 77.34) total: 10000 open/close in 133.56 seconds: 74.87 ops/second llite.lustre-ffff9676c3342800.statahead_batch_max=0 llite.lustre-ffff9676c3342800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.437s user 0m0.268s sys 0m22.730s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9676c3342800.statahead_batch_max=256 llite.lustre-ffff9676c3342800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9676c3342800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m40.189s user 0m0.251s sys 0m19.038s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9676c3342800.batch_stats= snapshot_time: 1784942435.062867207 secs.nsecs start_time: 1784942385.263033468 secs.nsecs elapsed_time: 49.799833739 secs.nsecs subreqs per batch batches %% cum %% 1: 246 58 58 2: 55 13 71 4: 37 8 80 8: 18 4 84 16: 12 2 87 32: 14 3 91 64: 2 0 91 128: 0 0 91 256: 35 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9676c3342800.statahead_batch_max=64 llite.lustre-ffff9676c3342800.statahead_max=128