== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:24:08 (1775877848) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=0 llite.lustre-ffff9aa887df1800.statahead_max=32 llite.lustre-ffff9aa887df1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.69 seconds: 143.90 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 101 real 0m0.564s user 0m0.003s sys 0m0.225s ls -l 100 files without statahead: 0 sec llite.lustre-ffff9aa887df1800.statahead_max=32 32 101 real 0m0.520s user 0m0.003s sys 0m0.321s ls -l 100 files with statahead: 1 sec statahead total: 97 statahead wrong: 2 agl total: 92 list_total: 32 fname_total: 60 hit_total: 22593 miss_total: 1114 total: 900 open/close in 1.66 seconds: 543.64 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 1001 real 0m6.238s user 0m0.022s sys 0m2.346s ls -l 1000 files without statahead: 6 sec llite.lustre-ffff9aa887df1800.statahead_max=32 32 1001 real 0m3.822s user 0m0.016s sys 0m1.745s ls -l 1000 files with statahead: 4 sec statahead total: 98 statahead wrong: 2 agl total: 93 list_total: 33 fname_total: 60 hit_total: 23592 miss_total: 1115 - open/close 4710 (time 1775877879.08 total 10.00 last 470.93) total: 9000 open/close in 19.46 seconds: 462.44 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 10001 real 0m59.060s user 0m0.243s sys 0m22.220s ls -l 10000 files without statahead: 59 sec llite.lustre-ffff9aa887df1800.statahead_max=32 32 10001 real 0m44.583s user 0m0.175s sys 0m20.422s ls -l 10000 files with statahead: 45 sec statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 60 hit_total: 33590 miss_total: 1117 ls -l done rm -r /mnt/lustre/d123ad.sanity: 115 seconds llite.lustre-ffff9aa887df1800.statahead_stats= statahead total: 99 statahead wrong: 2 agl total: 94 list_total: 34 fname_total: 60 hit_total: 33590 miss_total: 1117 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878121.347597455 secs.nsecs start_time: 1775877850.479253212 secs.nsecs elapsed_time: 270.868344243 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 4 1 1 8: 4 1 2 16: 204 57 60 32: 141 39 100 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878121.347654519 secs.nsecs start_time: 1775877850.479305857 secs.nsecs elapsed_time: 270.868348662 secs.nsecs subreqs per batch batches % cum % 1: 2 0 0 2: 1 0 0 4: 2 0 1 8: 6 1 3 16: 209 58 61 32: 135 38 100 - open/close 1418 (time 1775878132.70 total 10.00 last 141.80) - open/close 2948 (time 1775878142.70 total 20.00 last 152.97) - open/close 4565 (time 1775878152.70 total 30.00 last 161.70) - open/close 6806 (time 1775878162.70 total 40.00 last 224.09) - open/close 8209 (time 1775878172.70 total 50.01 last 140.24) - open/close 9584 (time 1775878182.71 total 60.02 last 137.37) total: 10000 open/close in 63.59 seconds: 157.26 ops/second llite.lustre-ffff9aa887df1800.statahead_batch_max=0 llite.lustre-ffff9aa887df1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.548s user 0m0.244s sys 0m24.325s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9aa887df1800.statahead_batch_max=32 llite.lustre-ffff9aa887df1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.789s user 0m0.217s sys 0m18.288s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878283.097592525 secs.nsecs start_time: 1775878227.338062505 secs.nsecs elapsed_time: 55.759530020 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878283.097650610 secs.nsecs start_time: 1775878227.338152582 secs.nsecs elapsed_time: 55.759498028 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=0 llite.lustre-ffff9aa887df1800.statahead_max=2048 llite.lustre-ffff9aa887df1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.32 seconds: 75.90 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 101 real 0m1.287s user 0m0.007s sys 0m0.296s ls -l 100 files without statahead: 2 sec llite.lustre-ffff9aa887df1800.statahead_max=2048 2048 101 real 0m0.443s user 0m0.005s sys 0m0.253s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 total: 900 open/close in 5.95 seconds: 151.27 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 1001 real 0m14.048s user 0m0.028s sys 0m2.850s ls -l 1000 files without statahead: 14 sec llite.lustre-ffff9aa887df1800.statahead_max=2048 2048 1001 real 0m3.665s user 0m0.018s sys 0m1.516s ls -l 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 - open/close 1817 (time 1775878486.94 total 10.00 last 181.67) total: 2000 open/close in 11.09 seconds: 180.42 ops/second llite.lustre-ffff9aa887df1800.statahead_max=0 3001 real 0m35.802s user 0m0.092s sys 0m8.307s ls -l 3000 files without statahead: 35 sec llite.lustre-ffff9aa887df1800.statahead_max=2048 2048 3001 real 0m9.242s user 0m0.057s sys 0m5.040s ls -l 3000 files with statahead: 10 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 14083 miss_total: 17 ls -l done rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff9aa887df1800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 14083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878568.036548371 secs.nsecs start_time: 1775878284.254221794 secs.nsecs elapsed_time: 283.782326577 secs.nsecs subreqs per batch batches % cum % 1: 83 72 72 2: 13 11 83 4: 4 3 86 8: 6 5 92 16: 0 0 92 32: 0 0 92 64: 1 0 93 128: 1 0 93 256: 7 6 100 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878568.036615236 secs.nsecs start_time: 1775878284.254369321 secs.nsecs elapsed_time: 283.782245915 secs.nsecs subreqs per batch batches % cum % 1: 83 70 70 2: 12 10 80 4: 6 5 85 8: 3 2 88 16: 2 1 89 32: 1 0 90 64: 2 1 92 128: 1 0 93 256: 8 6 100 - open/close 2168 (time 1775878579.13 total 10.00 last 216.75) - open/close 3662 (time 1775878589.14 total 20.01 last 149.36) - open/close 5323 (time 1775878599.14 total 30.01 last 166.05) - open/close 6631 (time 1775878609.14 total 40.01 last 130.74) - open/close 7820 (time 1775878619.16 total 50.03 last 118.74) - open/close 9302 (time 1775878629.17 total 60.04 last 148.06) total: 10000 open/close in 63.47 seconds: 157.55 ops/second llite.lustre-ffff9aa887df1800.statahead_batch_max=0 llite.lustre-ffff9aa887df1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m36.710s user 0m0.246s sys 0m28.309s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9aa887df1800.statahead_batch_max=256 llite.lustre-ffff9aa887df1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9aa887df1800.stats=clear mdc.lustre-MDT0001-mdc-ffff9aa887df1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.147s user 0m0.207s sys 0m19.596s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878709.400376454 secs.nsecs start_time: 1775878673.168528098 secs.nsecs elapsed_time: 36.231848356 secs.nsecs subreqs per batch batches % cum % 1: 177 55 55 2: 39 12 67 4: 30 9 77 8: 9 2 79 16: 5 1 81 32: 17 5 86 64: 7 2 89 128: 0 0 89 256: 35 10 100 mdc.lustre-MDT0001-mdc-ffff9aa887df1800.batch_stats= snapshot_time: 1775878709.400448978 secs.nsecs start_time: 1775878673.168670114 secs.nsecs elapsed_time: 36.231778864 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9aa887df1800.statahead_batch_max=64 llite.lustre-ffff9aa887df1800.statahead_max=128