== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:06:37 (1781222797) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=0 llite.lustre-ffff984182dc6800.statahead_max=32 llite.lustre-ffff984182dc6800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 3.25 seconds: 30.80 ops/second llite.lustre-ffff984182dc6800.statahead_max=0 101 real 0m3.714s user 0m0.009s sys 0m0.356s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff984182dc6800.statahead_max=32 llite.lustre-ffff984182dc6800.statahead_max=32 101 real 0m1.050s user 0m0.003s sys 0m0.188s 'ls -l' 100 files with statahead: 1 sec statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 31 fname_total: 70 hit_total: 3641 miss_total: 1340 swrong=2 ewrong=2 - open/close 732 (time 1781222830.35 total 10.01 last 73.14) total: 900 open/close in 12.49 seconds: 72.06 ops/second llite.lustre-ffff984182dc6800.statahead_max=0 1001 real 0m21.654s user 0m0.040s sys 0m3.165s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff984182dc6800.statahead_max=32 llite.lustre-ffff984182dc6800.statahead_max=32 1001 real 0m8.799s user 0m0.036s sys 0m1.562s 'ls -l' 1000 files with statahead: 9 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 32 fname_total: 70 hit_total: 4637 miss_total: 1344 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff984182dc6800.statahead_stats= statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 32 fname_total: 70 hit_total: 4637 miss_total: 1344 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781222895.286173151 secs.nsecs start_time: 1781222804.546677958 secs.nsecs elapsed_time: 90.739495193 secs.nsecs subreqs per batch batches % cum % 1: 2 5 5 2: 1 2 8 4: 1 2 11 8: 1 2 13 16: 24 66 80 32: 7 19 100 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781222895.286242555 secs.nsecs start_time: 1781222804.547559257 secs.nsecs elapsed_time: 90.738683298 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 13 33 43 32: 22 56 100 - open/close 864 (time 1781222907.38 total 10.00 last 86.38) - open/close 1731 (time 1781222917.39 total 20.01 last 86.63) - open/close 2578 (time 1781222927.39 total 30.02 last 84.66) - open/close 3341 (time 1781222937.40 total 40.02 last 76.23) - open/close 3848 (time 1781222947.41 total 50.03 last 50.68) - open/close 4553 (time 1781222957.42 total 60.04 last 70.42) - open/close 5349 (time 1781222967.42 total 70.05 last 79.55) - open/close 6136 (time 1781222977.44 total 80.06 last 78.60) - open/close 6980 (time 1781222987.44 total 90.06 last 84.39) - open/close 7828 (time 1781222997.45 total 100.07 last 84.70) - open/close 8766 (time 1781223007.45 total 110.08 last 93.76) - open/close 9613 (time 1781223017.46 total 120.08 last 84.65) total: 10000 open/close in 125.12 seconds: 79.92 ops/second llite.lustre-ffff984182dc6800.statahead_batch_max=0 llite.lustre-ffff984182dc6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.378s user 0m0.317s sys 0m17.288s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff984182dc6800.statahead_batch_max=32 llite.lustre-ffff984182dc6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m51.193s user 0m0.311s sys 0m15.537s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223217.628910180 secs.nsecs start_time: 1781223091.991322243 secs.nsecs elapsed_time: 125.637587937 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223217.628984139 secs.nsecs start_time: 1781223091.991413676 secs.nsecs elapsed_time: 125.637570463 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=0 llite.lustre-ffff984182dc6800.statahead_max=2048 llite.lustre-ffff984182dc6800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.67 seconds: 37.44 ops/second llite.lustre-ffff984182dc6800.statahead_max=0 101 real 0m2.573s user 0m0.006s sys 0m0.322s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff984182dc6800.statahead_max=2048 llite.lustre-ffff984182dc6800.statahead_max=2048 101 real 0m0.995s user 0m0.005s sys 0m0.220s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 811 (time 1781223520.74 total 10.01 last 81.04) total: 900 open/close in 11.15 seconds: 80.69 ops/second llite.lustre-ffff984182dc6800.statahead_max=0 1001 real 0m21.731s user 0m0.039s sys 0m3.098s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff984182dc6800.statahead_max=2048 llite.lustre-ffff984182dc6800.statahead_max=2048 1001 real 0m6.835s user 0m0.024s sys 0m1.601s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff984182dc6800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223581.806688307 secs.nsecs start_time: 1781223219.921832368 secs.nsecs elapsed_time: 361.884855939 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 12 12 4: 2 25 37 8: 2 25 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223581.806777735 secs.nsecs start_time: 1781223219.922339975 secs.nsecs elapsed_time: 361.884437760 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 2 25 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 904 (time 1781223594.00 total 10.00 last 90.40) - open/close 1743 (time 1781223604.01 total 20.01 last 83.79) - open/close 2561 (time 1781223614.03 total 30.03 last 81.69) - open/close 3350 (time 1781223624.04 total 40.04 last 78.78) - open/close 4113 (time 1781223634.05 total 50.05 last 76.27) - open/close 4782 (time 1781223644.06 total 60.06 last 66.82) - open/close 5394 (time 1781223654.07 total 70.07 last 61.13) - open/close 6203 (time 1781223664.08 total 80.08 last 80.84) - open/close 7047 (time 1781223674.09 total 90.09 last 84.33) - open/close 7743 (time 1781223684.09 total 100.09 last 69.56) - open/close 8605 (time 1781223694.09 total 110.09 last 86.18) - open/close 9548 (time 1781223704.10 total 120.10 last 94.29) total: 10000 open/close in 125.44 seconds: 79.72 ops/second llite.lustre-ffff984182dc6800.statahead_batch_max=0 llite.lustre-ffff984182dc6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.223s user 0m0.341s sys 0m16.532s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff984182dc6800.statahead_batch_max=256 llite.lustre-ffff984182dc6800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff984182dc6800.stats=clear mdc.lustre-MDT0001-mdc-ffff984182dc6800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.980s user 0m0.328s sys 0m17.447s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223856.854612746 secs.nsecs start_time: 1781223784.652655030 secs.nsecs elapsed_time: 72.201957716 secs.nsecs subreqs per batch batches % cum % 1: 206 48 48 2: 55 12 60 4: 67 15 76 8: 34 7 84 16: 12 2 87 32: 19 4 91 64: 2 0 92 128: 1 0 92 256: 33 7 100 mdc.lustre-MDT0001-mdc-ffff984182dc6800.batch_stats= snapshot_time: 1781223856.854686071 secs.nsecs start_time: 1781223784.653292950 secs.nsecs elapsed_time: 72.201393121 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff984182dc6800.statahead_batch_max=64 llite.lustre-ffff984182dc6800.statahead_max=128