== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:50:13 (1788270613) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=0 llite.lustre-ffff8ba547ff9800.statahead_max=32 llite.lustre-ffff8ba547ff9800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.56 seconds: 64.06 ops/second llite.lustre-ffff8ba547ff9800.statahead_max=0 101 real 0m1.350s user 0m0.007s sys 0m0.280s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8ba547ff9800.statahead_max=32 llite.lustre-ffff8ba547ff9800.statahead_max=32 101 real 0m0.605s user 0m0.002s sys 0m0.136s 'ls -l' 100 files with statahead: 0 sec statahead total: 125 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 80 hit_total: 4088 miss_total: 1278 swrong=2 ewrong=2 total: 900 open/close in 6.92 seconds: 130.08 ops/second llite.lustre-ffff8ba547ff9800.statahead_max=0 1001 real 0m13.545s user 0m0.036s sys 0m2.685s 'ls -l' 1000 files without statahead: 13 sec llite.lustre-ffff8ba547ff9800.statahead_max=32 llite.lustre-ffff8ba547ff9800.statahead_max=32 1001 real 0m5.152s user 0m0.024s sys 0m1.369s 'ls -l' 1000 files with statahead: 6 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 32 fname_total: 80 hit_total: 5083 miss_total: 1283 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 11 seconds llite.lustre-ffff8ba547ff9800.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 32 fname_total: 80 hit_total: 5083 miss_total: 1283 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270667.802326877 secs.nsecs start_time: 1788270616.584630868 secs.nsecs elapsed_time: 51.217696009 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 8 4: 2 5 13 8: 2 5 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270667.802396355 secs.nsecs start_time: 1788270616.584722869 secs.nsecs elapsed_time: 51.217673486 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 2 5 12 16: 12 30 42 32: 23 57 100 - open/close 1171 (time 1788270679.32 total 10.00 last 117.06) - open/close 2419 (time 1788270689.33 total 20.01 last 124.74) - open/close 3814 (time 1788270699.33 total 30.01 last 139.47) - open/close 5084 (time 1788270709.33 total 40.01 last 126.98) - open/close 6475 (time 1788270719.33 total 50.01 last 139.10) - open/close 7745 (time 1788270729.34 total 60.02 last 126.91) - open/close 8995 (time 1788270739.34 total 70.02 last 125.00) total: 10000 open/close in 75.93 seconds: 131.70 ops/second llite.lustre-ffff8ba547ff9800.statahead_batch_max=0 llite.lustre-ffff8ba547ff9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.773s user 0m0.250s sys 0m23.394s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ba547ff9800.statahead_batch_max=32 llite.lustre-ffff8ba547ff9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.387s user 0m0.201s sys 0m18.704s batched statahead RPC= hit count=9993 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270820.040338422 secs.nsecs start_time: 1788270779.372322828 secs.nsecs elapsed_time: 40.668015594 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270820.040396736 secs.nsecs start_time: 1788270779.372457465 secs.nsecs elapsed_time: 40.667939271 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=0 llite.lustre-ffff8ba547ff9800.statahead_max=2048 llite.lustre-ffff8ba547ff9800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 0.71 seconds: 141.79 ops/second llite.lustre-ffff8ba547ff9800.statahead_max=0 101 real 0m0.612s user 0m0.002s sys 0m0.223s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff8ba547ff9800.statahead_max=2048 llite.lustre-ffff8ba547ff9800.statahead_max=2048 101 real 0m0.334s user 0m0.003s sys 0m0.151s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10092 miss_total: 8 swrong=0 ewrong=0 total: 900 open/close in 1.84 seconds: 489.42 ops/second llite.lustre-ffff8ba547ff9800.statahead_max=0 1001 real 0m6.468s user 0m0.025s sys 0m2.243s 'ls -l' 1000 files without statahead: 7 sec llite.lustre-ffff8ba547ff9800.statahead_max=2048 llite.lustre-ffff8ba547ff9800.statahead_max=2048 1001 real 0m2.913s user 0m0.015s sys 0m1.072s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11091 miss_total: 9 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 4 seconds llite.lustre-ffff8ba547ff9800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11091 miss_total: 9 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270936.151443302 secs.nsecs start_time: 1788270820.988875178 secs.nsecs elapsed_time: 115.162568124 secs.nsecs subreqs per batch batches %% cum %% 1: 1 14 14 2: 0 0 14 4: 1 14 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788270936.151533124 secs.nsecs start_time: 1788270820.989011648 secs.nsecs elapsed_time: 115.162521476 secs.nsecs subreqs per batch batches %% cum %% 1: 1 14 14 2: 0 0 14 4: 1 14 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 3985 (time 1788270948.10 total 10.00 last 398.42) - open/close 7922 (time 1788270958.10 total 20.00 last 393.63) total: 10000 open/close in 24.93 seconds: 401.09 ops/second llite.lustre-ffff8ba547ff9800.statahead_batch_max=0 llite.lustre-ffff8ba547ff9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m27.491s user 0m0.166s sys 0m24.265s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ba547ff9800.statahead_batch_max=256 llite.lustre-ffff8ba547ff9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.stats=clear mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m26.317s user 0m0.191s sys 0m20.041s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788271023.447234404 secs.nsecs start_time: 1788270993.193228364 secs.nsecs elapsed_time: 30.254006040 secs.nsecs subreqs per batch batches %% cum %% 1: 245 62 62 2: 37 9 71 4: 41 10 82 8: 7 1 84 16: 3 0 84 32: 19 4 89 64: 6 1 91 128: 0 0 91 256: 34 8 100 mdc.lustre-MDT0001-mdc-ffff8ba547ff9800.batch_stats= snapshot_time: 1788271023.447292139 secs.nsecs start_time: 1788270993.193281356 secs.nsecs elapsed_time: 30.254010783 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8ba547ff9800.statahead_batch_max=64 llite.lustre-ffff8ba547ff9800.statahead_max=128