== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:07:13 (1779314833) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=0 llite.lustre-ffff890711fc9000.statahead_max=32 llite.lustre-ffff890711fc9000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.30 ops/second llite.lustre-ffff890711fc9000.statahead_max=0 101 real 0m2.213s user 0m0.005s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff890711fc9000.statahead_max=32 llite.lustre-ffff890711fc9000.statahead_max=32 101 real 0m1.066s user 0m0.004s sys 0m0.174s 'ls -l' 100 files with statahead: 1 sec statahead total: 125 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 94 hit_total: 4036 miss_total: 1387 swrong=2 ewrong=2 - open/close 754 (time 1779314860.84 total 10.01 last 75.33) total: 900 open/close in 12.04 seconds: 74.76 ops/second llite.lustre-ffff890711fc9000.statahead_max=0 1001 real 0m23.240s user 0m0.034s sys 0m3.057s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff890711fc9000.statahead_max=32 llite.lustre-ffff890711fc9000.statahead_max=32 1001 real 0m8.039s user 0m0.030s sys 0m1.551s 'ls -l' 1000 files with statahead: 8 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 32 fname_total: 94 hit_total: 5033 miss_total: 1390 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff890711fc9000.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 32 fname_total: 94 hit_total: 5033 miss_total: 1390 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats= snapshot_time: 1779314929.298847086 secs.nsecs start_time: 1779314837.930071979 secs.nsecs elapsed_time: 91.368775107 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 656 (time 1779314941.98 total 10.00 last 65.58) - open/close 1311 (time 1779314952.00 total 20.03 last 65.33) - open/close 2015 (time 1779314962.01 total 30.04 last 70.35) - open/close 2726 (time 1779314972.02 total 40.04 last 71.04) - open/close 3360 (time 1779314982.03 total 50.05 last 63.34) - open/close 3937 (time 1779314992.04 total 60.07 last 57.60) - open/close 4496 (time 1779315002.05 total 70.08 last 55.84) - open/close 5115 (time 1779315012.05 total 80.08 last 61.90) - open/close 5833 (time 1779315022.06 total 90.08 last 71.78) - open/close 6561 (time 1779315032.07 total 100.10 last 72.71) - open/close 7266 (time 1779315042.07 total 110.10 last 70.49) - open/close 7934 (time 1779315052.08 total 120.10 last 66.75) - open/close 8652 (time 1779315062.08 total 130.11 last 71.79) - open/close 9393 (time 1779315072.08 total 140.11 last 74.05) total: 10000 open/close in 148.44 seconds: 67.37 ops/second llite.lustre-ffff890711fc9000.statahead_batch_max=0 llite.lustre-ffff890711fc9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.276s user 0m0.323s sys 0m16.586s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff890711fc9000.statahead_batch_max=32 llite.lustre-ffff890711fc9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.284s user 0m0.277s sys 0m14.944s batched statahead RPC= hit count=9992 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats= snapshot_time: 1779315256.693224755 secs.nsecs start_time: 1779315147.877166558 secs.nsecs elapsed_time: 108.816058197 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=0 llite.lustre-ffff890711fc9000.statahead_max=2048 llite.lustre-ffff890711fc9000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=0 total: 100 open/close in 2.22 seconds: 45.07 ops/second llite.lustre-ffff890711fc9000.statahead_max=0 101 real 0m2.291s user 0m0.003s sys 0m0.331s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff890711fc9000.statahead_max=2048 llite.lustre-ffff890711fc9000.statahead_max=2048 101 real 0m0.701s user 0m0.004s sys 0m0.196s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10091 miss_total: 9 swrong=0 ewrong=0 - open/close 797 (time 1779315559.97 total 10.01 last 79.64) total: 900 open/close in 11.30 seconds: 79.65 ops/second llite.lustre-ffff890711fc9000.statahead_max=0 1001 real 0m19.277s user 0m0.030s sys 0m2.982s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff890711fc9000.statahead_max=2048 llite.lustre-ffff890711fc9000.statahead_max=2048 1001 real 0m4.999s user 0m0.028s sys 0m1.536s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11090 miss_total: 10 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 15 seconds llite.lustre-ffff890711fc9000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11090 miss_total: 10 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats= snapshot_time: 1779315610.627135402 secs.nsecs start_time: 1779315258.750514020 secs.nsecs elapsed_time: 351.876621382 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 822 (time 1779315622.70 total 10.00 last 82.18) - open/close 1633 (time 1779315632.70 total 20.01 last 81.07) - open/close 2573 (time 1779315642.71 total 30.01 last 93.94) - open/close 3446 (time 1779315652.71 total 40.02 last 87.26) - open/close 4295 (time 1779315662.72 total 50.02 last 84.86) - open/close 5122 (time 1779315672.72 total 60.02 last 82.69) - open/close 5891 (time 1779315682.73 total 70.04 last 76.80) - open/close 6684 (time 1779315692.75 total 80.05 last 79.17) - open/close 7423 (time 1779315702.75 total 90.05 last 73.88) - open/close 8227 (time 1779315712.75 total 100.06 last 80.38) - open/close 8977 (time 1779315722.75 total 110.06 last 75.00) - open/close 9842 (time 1779315732.75 total 120.06 last 86.50) total: 10000 open/close in 121.75 seconds: 82.14 ops/second llite.lustre-ffff890711fc9000.statahead_batch_max=0 llite.lustre-ffff890711fc9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.055s user 0m0.307s sys 0m19.856s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff890711fc9000.statahead_batch_max=256 llite.lustre-ffff890711fc9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff890711fc9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.752s user 0m0.327s sys 0m17.207s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff890711fc9000.batch_stats= snapshot_time: 1779315839.210312055 secs.nsecs start_time: 1779315785.027356926 secs.nsecs elapsed_time: 54.182955129 secs.nsecs subreqs per batch batches % cum % 1: 287 59 59 2: 49 10 69 4: 51 10 80 8: 28 5 85 16: 20 4 90 32: 12 2 92 64: 2 0 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff890711fc9000.statahead_batch_max=64 llite.lustre-ffff890711fc9000.statahead_max=128