== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:12:38 (1779991958) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=0 llite.lustre-ffff970410840000.statahead_max=32 llite.lustre-ffff970410840000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.79 ops/second llite.lustre-ffff970410840000.statahead_max=0 101 real 0m1.863s user 0m0.007s sys 0m0.335s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff970410840000.statahead_max=32 llite.lustre-ffff970410840000.statahead_max=32 101 real 0m0.866s user 0m0.004s sys 0m0.146s 'ls -l' 100 files with statahead: 1 sec statahead total: 124 statahead wrong: 1 agl total: 118 list_total: 42 fname_total: 67 hit_total: 3903 miss_total: 255 swrong=1 ewrong=1 - open/close 715 (time 1779991984.67 total 10.01 last 71.45) total: 900 open/close in 12.44 seconds: 72.35 ops/second llite.lustre-ffff970410840000.statahead_max=0 1001 real 0m19.032s user 0m0.025s sys 0m3.137s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff970410840000.statahead_max=32 llite.lustre-ffff970410840000.statahead_max=32 1001 real 0m10.518s user 0m0.029s sys 0m1.481s 'ls -l' 1000 files with statahead: 11 sec statahead total: 125 statahead wrong: 1 agl total: 119 list_total: 43 fname_total: 67 hit_total: 4902 miss_total: 256 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff970410840000.statahead_stats= statahead total: 125 statahead wrong: 1 agl total: 119 list_total: 43 fname_total: 67 hit_total: 4902 miss_total: 256 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats= snapshot_time: 1779992050.670811001 secs.nsecs start_time: 1779991963.701971121 secs.nsecs elapsed_time: 86.968839880 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 732 (time 1779992063.25 total 10.00 last 73.19) - open/close 1518 (time 1779992073.25 total 20.00 last 78.58) - open/close 2212 (time 1779992083.26 total 30.01 last 69.33) - open/close 3072 (time 1779992093.27 total 40.02 last 85.92) - open/close 3967 (time 1779992103.27 total 50.03 last 89.44) - open/close 4795 (time 1779992113.28 total 60.03 last 82.77) - open/close 5469 (time 1779992123.28 total 70.04 last 67.36) - open/close 6259 (time 1779992133.29 total 80.05 last 78.95) - open/close 7096 (time 1779992143.30 total 90.05 last 83.64) - open/close 7910 (time 1779992153.30 total 100.06 last 81.35) - open/close 8711 (time 1779992163.31 total 110.07 last 80.04) - open/close 9550 (time 1779992173.31 total 120.07 last 83.89) total: 10000 open/close in 125.92 seconds: 79.41 ops/second llite.lustre-ffff970410840000.statahead_batch_max=0 llite.lustre-ffff970410840000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.506s user 0m0.297s sys 0m15.065s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff970410840000.statahead_batch_max=32 llite.lustre-ffff970410840000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.424s user 0m0.310s sys 0m14.264s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats= snapshot_time: 1779992373.031771723 secs.nsecs start_time: 1779992251.610663698 secs.nsecs elapsed_time: 121.421108025 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=0 llite.lustre-ffff970410840000.statahead_max=2048 llite.lustre-ffff970410840000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.83 ops/second llite.lustre-ffff970410840000.statahead_max=0 101 real 0m2.343s user 0m0.003s sys 0m0.350s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff970410840000.statahead_max=2048 llite.lustre-ffff970410840000.statahead_max=2048 101 real 0m0.695s user 0m0.003s sys 0m0.144s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 695 (time 1779992693.15 total 10.00 last 69.48) total: 900 open/close in 12.76 seconds: 70.51 ops/second llite.lustre-ffff970410840000.statahead_max=0 1001 real 0m24.716s user 0m0.026s sys 0m3.195s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff970410840000.statahead_max=2048 llite.lustre-ffff970410840000.statahead_max=2048 1001 real 0m7.191s user 0m0.030s sys 0m1.428s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff970410840000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats= snapshot_time: 1779992763.424623223 secs.nsecs start_time: 1779992375.835787810 secs.nsecs elapsed_time: 387.588835413 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 699 (time 1779992776.30 total 10.01 last 69.84) - open/close 1316 (time 1779992786.32 total 20.02 last 61.63) - open/close 2018 (time 1779992796.32 total 30.03 last 70.14) - open/close 2731 (time 1779992806.33 total 40.04 last 71.24) - open/close 3480 (time 1779992816.37 total 50.08 last 74.58) - open/close 4278 (time 1779992826.39 total 60.10 last 79.65) - open/close 5070 (time 1779992836.40 total 70.10 last 79.15) - open/close 5831 (time 1779992846.41 total 80.11 last 76.04) - open/close 6569 (time 1779992856.44 total 90.14 last 73.60) - open/close 7275 (time 1779992866.44 total 100.14 last 70.58) - open/close 8197 (time 1779992876.44 total 110.15 last 92.17) - open/close 8930 (time 1779992886.45 total 120.16 last 73.21) - open/close 9656 (time 1779992896.46 total 130.17 last 72.55) total: 10000 open/close in 135.38 seconds: 73.86 ops/second llite.lustre-ffff970410840000.statahead_batch_max=0 llite.lustre-ffff970410840000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.323s user 0m0.342s sys 0m15.364s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff970410840000.statahead_batch_max=256 llite.lustre-ffff970410840000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff970410840000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.349s user 0m0.304s sys 0m15.559s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff970410840000.batch_stats= snapshot_time: 1779993063.548869146 secs.nsecs start_time: 1779992986.616925591 secs.nsecs elapsed_time: 76.931943555 secs.nsecs subreqs per batch batches % cum % 1: 184 47 47 2: 70 18 65 4: 43 11 76 8: 22 5 82 16: 15 3 86 32: 17 4 90 64: 2 0 90 128: 1 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff970410840000.statahead_batch_max=64 llite.lustre-ffff970410840000.statahead_max=128