== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:02:43 (1790924563) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=0 llite.lustre-ffff97b29881f000.statahead_max=32 llite.lustre-ffff97b29881f000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=0 total: 100 open/close in 2.65 seconds: 37.79 ops/second llite.lustre-ffff97b29881f000.statahead_max=0 101 real 0m1.953s user 0m0.006s sys 0m0.284s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97b29881f000.statahead_max=32 llite.lustre-ffff97b29881f000.statahead_max=32 101 real 0m0.815s user 0m0.004s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 30 fname_total: 82 hit_total: 4104 miss_total: 1671 swrong=2 ewrong=2 - open/close 723 (time 1790924587.87 total 10.01 last 72.23) total: 900 open/close in 12.63 seconds: 71.27 ops/second llite.lustre-ffff97b29881f000.statahead_max=0 1001 real 0m20.737s user 0m0.027s sys 0m2.750s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff97b29881f000.statahead_max=32 llite.lustre-ffff97b29881f000.statahead_max=32 1001 real 0m10.325s user 0m0.034s sys 0m1.416s 'ls -l' 1000 files with statahead: 11 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 82 hit_total: 5101 miss_total: 1674 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff97b29881f000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 82 hit_total: 5101 miss_total: 1674 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats= snapshot_time: 1790924661.178892375 secs.nsecs start_time: 1790924567.187971952 secs.nsecs elapsed_time: 93.990920423 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 585 (time 1790924674.57 total 10.02 last 58.36) - open/close 1213 (time 1790924684.59 total 20.05 last 62.67) - open/close 1902 (time 1790924694.59 total 30.05 last 68.85) - open/close 2473 (time 1790924704.60 total 40.06 last 57.05) - open/close 3092 (time 1790924714.63 total 50.09 last 61.74) - open/close 3667 (time 1790924724.67 total 60.13 last 57.27) - open/close 4286 (time 1790924734.69 total 70.14 last 61.80) - open/close 4894 (time 1790924744.70 total 80.16 last 60.73) - open/close 5504 (time 1790924754.70 total 90.16 last 61.00) - open/close 6208 (time 1790924764.71 total 100.17 last 70.29) - open/close 6929 (time 1790924774.72 total 110.18 last 72.03) - open/close 7632 (time 1790924784.74 total 120.20 last 70.17) - open/close 8360 (time 1790924794.74 total 130.20 last 72.77) - open/close 9091 (time 1790924804.75 total 140.21 last 73.09) - open/close 9769 (time 1790924814.76 total 150.22 last 67.73) total: 10000 open/close in 153.11 seconds: 65.31 ops/second llite.lustre-ffff97b29881f000.statahead_batch_max=0 llite.lustre-ffff97b29881f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.188s user 0m0.315s sys 0m15.672s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97b29881f000.statahead_batch_max=32 llite.lustre-ffff97b29881f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m37.681s user 0m0.310s sys 0m13.266s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats= snapshot_time: 1790924990.761485790 secs.nsecs start_time: 1790924881.806204952 secs.nsecs elapsed_time: 108.955280838 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=0 llite.lustre-ffff97b29881f000.statahead_max=2048 llite.lustre-ffff97b29881f000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=0 total: 100 open/close in 2.48 seconds: 40.35 ops/second llite.lustre-ffff97b29881f000.statahead_max=0 101 real 0m2.585s user 0m0.005s sys 0m0.320s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff97b29881f000.statahead_max=2048 llite.lustre-ffff97b29881f000.statahead_max=2048 101 real 0m1.148s user 0m0.003s sys 0m0.153s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 694 (time 1790925317.16 total 10.01 last 69.30) total: 900 open/close in 13.37 seconds: 67.32 ops/second llite.lustre-ffff97b29881f000.statahead_max=0 1001 real 0m22.972s user 0m0.032s sys 0m2.832s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff97b29881f000.statahead_max=2048 llite.lustre-ffff97b29881f000.statahead_max=2048 1001 real 0m7.958s user 0m0.029s sys 0m1.383s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff97b29881f000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats= snapshot_time: 1790925387.978390566 secs.nsecs start_time: 1790924993.596921392 secs.nsecs elapsed_time: 394.381469174 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 677 (time 1790925400.86 total 10.01 last 67.65) - open/close 1434 (time 1790925410.86 total 20.01 last 75.66) - open/close 2166 (time 1790925420.88 total 30.03 last 73.08) - open/close 2893 (time 1790925430.89 total 40.04 last 72.62) - open/close 3590 (time 1790925440.90 total 50.05 last 69.63) - open/close 4303 (time 1790925450.94 total 60.08 last 71.06) - open/close 4970 (time 1790925460.94 total 70.09 last 66.67) - open/close 5721 (time 1790925470.94 total 80.09 last 75.08) - open/close 6461 (time 1790925480.95 total 90.10 last 73.94) - open/close 7178 (time 1790925490.95 total 100.10 last 71.70) - open/close 7857 (time 1790925500.96 total 110.10 last 67.86) - open/close 8572 (time 1790925510.96 total 120.11 last 71.47) - open/close 9298 (time 1790925520.97 total 130.12 last 72.53) total: 10000 open/close in 139.97 seconds: 71.44 ops/second llite.lustre-ffff97b29881f000.statahead_batch_max=0 llite.lustre-ffff97b29881f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.141s user 0m0.297s sys 0m15.071s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97b29881f000.statahead_batch_max=256 llite.lustre-ffff97b29881f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97b29881f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.495s user 0m0.320s sys 0m14.657s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff97b29881f000.batch_stats= snapshot_time: 1790925659.866270674 secs.nsecs start_time: 1790925597.596430301 secs.nsecs elapsed_time: 62.269840373 secs.nsecs subreqs per batch batches %% cum %% 1: 196 53 53 2: 51 13 67 4: 29 7 75 8: 19 5 80 16: 12 3 83 32: 19 5 88 64: 6 1 90 128: 1 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff97b29881f000.statahead_batch_max=64 llite.lustre-ffff97b29881f000.statahead_max=128