== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:26:34 (1787858794) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=0 llite.lustre-ffff9f04f0bca000.statahead_max=32 llite.lustre-ffff9f04f0bca000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=0 total: 100 open/close in 2.94 seconds: 33.96 ops/second llite.lustre-ffff9f04f0bca000.statahead_max=0 101 real 0m2.577s user 0m0.008s sys 0m0.300s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f04f0bca000.statahead_max=32 llite.lustre-ffff9f04f0bca000.statahead_max=32 101 real 0m1.245s user 0m0.005s sys 0m0.152s 'ls -l' 100 files with statahead: 1 sec statahead total: 112 statahead wrong: 2 agl total: 106 list_total: 30 fname_total: 63 hit_total: 4123 miss_total: 1587 swrong=2 ewrong=2 - open/close 628 (time 1787858822.49 total 10.01 last 62.73) total: 900 open/close in 14.13 seconds: 63.70 ops/second llite.lustre-ffff9f04f0bca000.statahead_max=0 1001 real 0m22.174s user 0m0.035s sys 0m3.009s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9f04f0bca000.statahead_max=32 llite.lustre-ffff9f04f0bca000.statahead_max=32 1001 real 0m9.926s user 0m0.024s sys 0m1.298s 'ls -l' 1000 files with statahead: 10 sec statahead total: 113 statahead wrong: 2 agl total: 107 list_total: 31 fname_total: 63 hit_total: 5120 miss_total: 1590 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9f04f0bca000.statahead_stats= statahead total: 113 statahead wrong: 2 agl total: 107 list_total: 31 fname_total: 63 hit_total: 5120 miss_total: 1590 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats= snapshot_time: 1787858893.030013138 secs.nsecs start_time: 1787858799.264282445 secs.nsecs elapsed_time: 93.765730693 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 711 (time 1787858905.75 total 10.01 last 71.06) - open/close 1421 (time 1787858915.75 total 20.01 last 70.99) - open/close 2082 (time 1787858925.76 total 30.02 last 66.02) - open/close 2619 (time 1787858935.77 total 40.03 last 53.64) - open/close 3242 (time 1787858945.77 total 50.03 last 62.29) - open/close 3933 (time 1787858955.79 total 60.05 last 69.01) - open/close 4558 (time 1787858965.80 total 70.06 last 62.43) - open/close 5196 (time 1787858975.81 total 80.07 last 63.74) - open/close 5874 (time 1787858985.82 total 90.08 last 67.73) - open/close 6541 (time 1787858995.83 total 100.09 last 66.64) - open/close 7164 (time 1787859005.83 total 110.09 last 62.25) - open/close 7820 (time 1787859015.84 total 120.09 last 65.59) - open/close 8608 (time 1787859025.84 total 130.10 last 78.76) - open/close 9209 (time 1787859035.85 total 140.11 last 60.06) - open/close 9881 (time 1787859045.86 total 150.12 last 67.12) total: 10000 open/close in 151.76 seconds: 65.89 ops/second llite.lustre-ffff9f04f0bca000.statahead_batch_max=0 llite.lustre-ffff9f04f0bca000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.487s user 0m0.352s sys 0m16.175s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f04f0bca000.statahead_batch_max=32 llite.lustre-ffff9f04f0bca000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m34.968s user 0m0.330s sys 0m14.140s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats= snapshot_time: 1787859221.701967645 secs.nsecs start_time: 1787859114.681885393 secs.nsecs elapsed_time: 107.020082252 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=0 llite.lustre-ffff9f04f0bca000.statahead_max=2048 llite.lustre-ffff9f04f0bca000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=0 total: 100 open/close in 2.89 seconds: 34.62 ops/second llite.lustre-ffff9f04f0bca000.statahead_max=0 101 real 0m2.209s user 0m0.003s sys 0m0.313s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f04f0bca000.statahead_max=2048 llite.lustre-ffff9f04f0bca000.statahead_max=2048 101 real 0m0.809s user 0m0.006s sys 0m0.193s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 677 (time 1787859535.95 total 10.01 last 67.66) total: 900 open/close in 13.47 seconds: 66.81 ops/second llite.lustre-ffff9f04f0bca000.statahead_max=0 1001 real 0m23.199s user 0m0.035s sys 0m3.078s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9f04f0bca000.statahead_max=2048 llite.lustre-ffff9f04f0bca000.statahead_max=2048 1001 real 0m6.364s user 0m0.028s sys 0m1.452s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9f04f0bca000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats= snapshot_time: 1787859602.507809121 secs.nsecs start_time: 1787859224.566787195 secs.nsecs elapsed_time: 377.941021926 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 618 (time 1787859615.31 total 10.00 last 61.78) - open/close 1242 (time 1787859625.31 total 20.01 last 62.36) - open/close 1870 (time 1787859635.31 total 30.01 last 62.79) - open/close 2522 (time 1787859645.34 total 40.03 last 65.05) - open/close 3141 (time 1787859655.34 total 50.04 last 61.85) - open/close 3759 (time 1787859665.36 total 60.06 last 61.71) - open/close 4472 (time 1787859675.36 total 70.06 last 71.27) - open/close 5125 (time 1787859685.38 total 80.08 last 65.16) - open/close 5780 (time 1787859695.39 total 90.09 last 65.45) - open/close 6457 (time 1787859705.40 total 100.10 last 67.64) - open/close 7099 (time 1787859715.41 total 110.11 last 64.14) - open/close 7735 (time 1787859725.43 total 120.12 last 63.50) - open/close 8324 (time 1787859735.44 total 130.13 last 58.84) - open/close 8927 (time 1787859745.46 total 140.15 last 60.18) - open/close 9609 (time 1787859755.46 total 150.16 last 68.15) total: 10000 open/close in 156.05 seconds: 64.08 ops/second llite.lustre-ffff9f04f0bca000.statahead_batch_max=0 llite.lustre-ffff9f04f0bca000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.463s user 0m0.342s sys 0m15.907s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f04f0bca000.statahead_batch_max=256 llite.lustre-ffff9f04f0bca000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.167s user 0m0.273s sys 0m15.745s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f04f0bca000.batch_stats= snapshot_time: 1787859883.012420689 secs.nsecs start_time: 1787859829.313663205 secs.nsecs elapsed_time: 53.698757484 secs.nsecs subreqs per batch batches %% cum %% 1: 100 46 46 2: 21 9 55 4: 18 8 64 8: 15 6 70 16: 5 2 73 32: 14 6 79 64: 7 3 82 128: 2 0 83 256: 35 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f04f0bca000.statahead_batch_max=64 llite.lustre-ffff9f04f0bca000.statahead_max=128