== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:32:18 (1788593538) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=0 llite.lustre-ffff9be9e10b1800.statahead_max=32 llite.lustre-ffff9be9e10b1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.33 ops/second llite.lustre-ffff9be9e10b1800.statahead_max=0 101 real 0m2.220s user 0m0.008s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9be9e10b1800.statahead_max=32 llite.lustre-ffff9be9e10b1800.statahead_max=32 101 real 0m0.804s user 0m0.000s sys 0m0.178s 'ls -l' 100 files with statahead: 1 sec statahead total: 116 statahead wrong: 2 agl total: 110 list_total: 30 fname_total: 83 hit_total: 4388 miss_total: 1289 swrong=2 ewrong=2 - open/close 842 (time 1788593566.36 total 10.00 last 84.16) total: 900 open/close in 10.78 seconds: 83.52 ops/second llite.lustre-ffff9be9e10b1800.statahead_max=0 1001 real 0m20.121s user 0m0.030s sys 0m2.687s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9be9e10b1800.statahead_max=32 llite.lustre-ffff9be9e10b1800.statahead_max=32 1001 real 0m8.584s user 0m0.017s sys 0m1.295s 'ls -l' 1000 files with statahead: 9 sec statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 83 hit_total: 5386 miss_total: 1291 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9be9e10b1800.statahead_stats= statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 83 hit_total: 5386 miss_total: 1291 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats= snapshot_time: 1788593629.479657904 secs.nsecs start_time: 1788593544.632258650 secs.nsecs elapsed_time: 84.847399254 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 775 (time 1788593642.12 total 10.01 last 77.42) - open/close 1624 (time 1788593652.13 total 20.02 last 84.83) - open/close 2507 (time 1788593662.14 total 30.03 last 88.20) - open/close 3370 (time 1788593672.15 total 40.04 last 86.20) - open/close 4264 (time 1788593682.15 total 50.05 last 89.37) - open/close 5130 (time 1788593692.16 total 60.05 last 86.54) - open/close 6030 (time 1788593702.16 total 70.06 last 89.99) - open/close 6719 (time 1788593712.17 total 80.07 last 68.83) - open/close 7521 (time 1788593722.20 total 90.09 last 80.02) - open/close 8312 (time 1788593732.20 total 100.09 last 79.08) - open/close 9218 (time 1788593742.21 total 110.10 last 90.48) total: 10000 open/close in 119.87 seconds: 83.43 ops/second llite.lustre-ffff9be9e10b1800.statahead_batch_max=0 llite.lustre-ffff9be9e10b1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.427s user 0m0.287s sys 0m13.277s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9be9e10b1800.statahead_batch_max=32 llite.lustre-ffff9be9e10b1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m36.044s user 0m0.287s sys 0m12.763s batched statahead RPC= hit count=9972 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats= snapshot_time: 1788593927.190342399 secs.nsecs start_time: 1788593821.549256109 secs.nsecs elapsed_time: 105.641086290 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=0 llite.lustre-ffff9be9e10b1800.statahead_max=2048 llite.lustre-ffff9be9e10b1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.76 ops/second llite.lustre-ffff9be9e10b1800.statahead_max=0 101 real 0m1.837s user 0m0.004s sys 0m0.288s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9be9e10b1800.statahead_max=2048 llite.lustre-ffff9be9e10b1800.statahead_max=2048 101 real 0m0.965s user 0m0.003s sys 0m0.162s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10071 miss_total: 29 swrong=0 ewrong=0 - open/close 805 (time 1788594209.81 total 10.00 last 80.46) total: 900 open/close in 11.34 seconds: 79.39 ops/second llite.lustre-ffff9be9e10b1800.statahead_max=0 1001 real 0m20.999s user 0m0.040s sys 0m2.721s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9be9e10b1800.statahead_max=2048 llite.lustre-ffff9be9e10b1800.statahead_max=2048 1001 real 0m7.710s user 0m0.027s sys 0m1.260s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9be9e10b1800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats= snapshot_time: 1788594271.165598830 secs.nsecs start_time: 1788593929.431227959 secs.nsecs elapsed_time: 341.734370871 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 774 (time 1788594283.54 total 10.00 last 77.38) - open/close 1625 (time 1788594293.54 total 20.01 last 85.06) - open/close 2438 (time 1788594303.55 total 30.01 last 81.24) - open/close 3294 (time 1788594313.55 total 40.02 last 85.59) - open/close 4147 (time 1788594323.56 total 50.02 last 85.27) - open/close 4970 (time 1788594333.56 total 60.02 last 82.25) - open/close 5898 (time 1788594343.57 total 70.03 last 92.73) - open/close 6783 (time 1788594353.59 total 80.05 last 88.34) - open/close 7577 (time 1788594363.59 total 90.05 last 79.38) - open/close 8442 (time 1788594373.60 total 100.06 last 86.43) - open/close 9253 (time 1788594383.60 total 110.06 last 81.08) total: 10000 open/close in 119.06 seconds: 83.99 ops/second llite.lustre-ffff9be9e10b1800.statahead_batch_max=0 llite.lustre-ffff9be9e10b1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.333s user 0m0.279s sys 0m13.166s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9be9e10b1800.statahead_batch_max=256 llite.lustre-ffff9be9e10b1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.103s user 0m0.288s sys 0m15.048s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9be9e10b1800.batch_stats= snapshot_time: 1788594514.663037672 secs.nsecs start_time: 1788594462.107222978 secs.nsecs elapsed_time: 52.555814694 secs.nsecs subreqs per batch batches %% cum %% 1: 128 47 47 2: 41 15 62 4: 27 9 72 8: 8 2 75 16: 12 4 79 32: 10 3 83 64: 10 3 86 128: 2 0 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9be9e10b1800.statahead_batch_max=64 llite.lustre-ffff9be9e10b1800.statahead_max=128