== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:24:11 (1787397851) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=0 llite.lustre-ffff897dd1327800.statahead_max=32 llite.lustre-ffff897dd1327800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.53 seconds: 39.48 ops/second llite.lustre-ffff897dd1327800.statahead_max=0 101 real 0m2.601s user 0m0.006s sys 0m0.330s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff897dd1327800.statahead_max=32 llite.lustre-ffff897dd1327800.statahead_max=32 101 real 0m1.255s user 0m0.005s sys 0m0.220s 'ls -l' 100 files with statahead: 1 sec statahead total: 146 statahead wrong: 2 agl total: 140 list_total: 31 fname_total: 102 hit_total: 3674 miss_total: 1406 swrong=2 ewrong=2 - open/close 800 (time 1787397878.40 total 10.01 last 79.93) total: 900 open/close in 11.29 seconds: 79.73 ops/second llite.lustre-ffff897dd1327800.statahead_max=0 1001 real 0m23.482s user 0m0.032s sys 0m3.092s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff897dd1327800.statahead_max=32 llite.lustre-ffff897dd1327800.statahead_max=32 1001 real 0m9.538s user 0m0.038s sys 0m1.738s 'ls -l' 1000 files with statahead: 9 sec statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 32 fname_total: 102 hit_total: 4673 miss_total: 1407 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff897dd1327800.statahead_stats= statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 32 fname_total: 102 hit_total: 4673 miss_total: 1407 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787397947.483956687 secs.nsecs start_time: 1787397855.556401738 secs.nsecs elapsed_time: 91.927554949 secs.nsecs subreqs per batch batches %% cum %% 1: 5 12 12 2: 2 5 17 4: 1 2 20 8: 1 2 23 16: 22 56 79 32: 8 20 100 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787397947.484044120 secs.nsecs start_time: 1787397855.556816886 secs.nsecs elapsed_time: 91.927227234 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 1 2 7 4: 0 0 7 8: 4 9 17 16: 13 31 48 32: 21 51 100 - open/close 732 (time 1787397959.94 total 10.01 last 73.15) - open/close 1545 (time 1787397969.94 total 20.01 last 81.27) - open/close 2366 (time 1787397979.96 total 30.02 last 81.99) - open/close 3181 (time 1787397989.96 total 40.03 last 81.47) - open/close 4012 (time 1787397999.97 total 50.03 last 83.03) - open/close 4883 (time 1787398009.98 total 60.05 last 86.95) - open/close 5678 (time 1787398019.99 total 70.06 last 79.44) - open/close 6440 (time 1787398030.00 total 80.06 last 76.16) - open/close 7206 (time 1787398040.01 total 90.07 last 76.53) - open/close 7968 (time 1787398050.02 total 100.09 last 76.09) - open/close 8774 (time 1787398060.02 total 110.09 last 80.57) - open/close 9505 (time 1787398070.03 total 120.10 last 73.04) total: 10000 open/close in 126.64 seconds: 78.96 ops/second llite.lustre-ffff897dd1327800.statahead_batch_max=0 llite.lustre-ffff897dd1327800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.353s user 0m0.411s sys 0m19.375s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff897dd1327800.statahead_batch_max=32 llite.lustre-ffff897dd1327800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m43.318s user 0m0.340s sys 0m17.437s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398272.316145274 secs.nsecs start_time: 1787398156.751852400 secs.nsecs elapsed_time: 115.564292874 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398272.316285583 secs.nsecs start_time: 1787398156.751946937 secs.nsecs elapsed_time: 115.564338646 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=0 llite.lustre-ffff897dd1327800.statahead_max=2048 llite.lustre-ffff897dd1327800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 2.76 seconds: 36.27 ops/second llite.lustre-ffff897dd1327800.statahead_max=0 101 real 0m2.889s user 0m0.006s sys 0m0.353s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff897dd1327800.statahead_max=2048 llite.lustre-ffff897dd1327800.statahead_max=2048 101 real 0m0.957s user 0m0.005s sys 0m0.216s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 793 (time 1787398602.71 total 10.01 last 79.23) total: 900 open/close in 11.30 seconds: 79.66 ops/second llite.lustre-ffff897dd1327800.statahead_max=0 1001 real 0m20.890s user 0m0.039s sys 0m3.365s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff897dd1327800.statahead_max=2048 llite.lustre-ffff897dd1327800.statahead_max=2048 1001 real 0m7.636s user 0m0.035s sys 0m2.007s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff897dd1327800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398664.099753559 secs.nsecs start_time: 1787398274.504785887 secs.nsecs elapsed_time: 389.594967672 secs.nsecs subreqs per batch batches %% cum %% 1: 3 37 37 2: 2 25 62 4: 1 12 75 8: 0 0 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398664.099892441 secs.nsecs start_time: 1787398274.504997487 secs.nsecs elapsed_time: 389.594894954 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 4 44 44 16: 1 11 55 32: 0 0 55 64: 2 22 77 128: 0 0 77 256: 2 22 100 - open/close 726 (time 1787398676.64 total 10.00 last 72.58) - open/close 1501 (time 1787398686.64 total 20.01 last 77.48) - open/close 2275 (time 1787398696.64 total 30.01 last 77.40) - open/close 3017 (time 1787398706.65 total 40.01 last 74.18) - open/close 3619 (time 1787398716.66 total 50.02 last 60.12) - open/close 4398 (time 1787398726.67 total 60.03 last 77.84) - open/close 5183 (time 1787398736.67 total 70.03 last 78.49) - open/close 5966 (time 1787398746.68 total 80.04 last 78.21) - open/close 6787 (time 1787398756.69 total 90.05 last 82.02) - open/close 7512 (time 1787398766.69 total 100.05 last 72.49) - open/close 8304 (time 1787398776.70 total 110.06 last 79.18) - open/close 9109 (time 1787398786.70 total 120.06 last 80.48) - open/close 9887 (time 1787398796.70 total 130.06 last 77.78) total: 10000 open/close in 131.71 seconds: 75.93 ops/second llite.lustre-ffff897dd1327800.statahead_batch_max=0 llite.lustre-ffff897dd1327800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.580s user 0m0.336s sys 0m20.115s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff897dd1327800.statahead_batch_max=256 llite.lustre-ffff897dd1327800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff897dd1327800.stats=clear mdc.lustre-MDT0001-mdc-ffff897dd1327800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.798s user 0m0.313s sys 0m19.251s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398934.842218350 secs.nsecs start_time: 1787398865.522007219 secs.nsecs elapsed_time: 69.320211131 secs.nsecs subreqs per batch batches %% cum %% 1: 259 60 60 2: 48 11 71 4: 34 7 79 8: 15 3 83 16: 12 2 85 32: 18 4 90 64: 8 1 92 128: 1 0 92 256: 33 7 100 mdc.lustre-MDT0001-mdc-ffff897dd1327800.batch_stats= snapshot_time: 1787398934.842299245 secs.nsecs start_time: 1787398865.522142149 secs.nsecs elapsed_time: 69.320157096 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff897dd1327800.statahead_batch_max=64 llite.lustre-ffff897dd1327800.statahead_max=128