== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:22:57 (1781230977) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=0 llite.lustre-ffff9f1f8478e000.statahead_max=32 llite.lustre-ffff9f1f8478e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=0 total: 100 open/close in 2.81 seconds: 35.59 ops/second llite.lustre-ffff9f1f8478e000.statahead_max=0 101 real 0m2.376s user 0m0.001s sys 0m0.331s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f1f8478e000.statahead_max=32 llite.lustre-ffff9f1f8478e000.statahead_max=32 101 real 0m1.581s user 0m0.001s sys 0m0.202s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 31 fname_total: 88 hit_total: 4110 miss_total: 1290 swrong=2 ewrong=2 - open/close 679 (time 1781231005.79 total 10.01 last 67.84) total: 900 open/close in 13.57 seconds: 66.31 ops/second llite.lustre-ffff9f1f8478e000.statahead_max=0 1001 real 0m24.226s user 0m0.038s sys 0m3.119s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9f1f8478e000.statahead_max=32 llite.lustre-ffff9f1f8478e000.statahead_max=32 1001 real 0m13.080s user 0m0.026s sys 0m1.566s 'ls -l' 1000 files with statahead: 13 sec statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 32 fname_total: 88 hit_total: 5107 miss_total: 1293 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9f1f8478e000.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 32 fname_total: 88 hit_total: 5107 miss_total: 1293 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats= snapshot_time: 1781231083.607812584 secs.nsecs start_time: 1781230982.623489041 secs.nsecs elapsed_time: 100.984323543 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 602 (time 1781231096.51 total 10.00 last 60.17) - open/close 1257 (time 1781231106.53 total 20.02 last 65.37) - open/close 1896 (time 1781231116.53 total 30.03 last 63.89) - open/close 2473 (time 1781231126.57 total 40.07 last 57.47) - open/close 2995 (time 1781231136.58 total 50.07 last 52.18) - open/close 3608 (time 1781231146.59 total 60.08 last 61.25) - open/close 4121 (time 1781231156.59 total 70.08 last 51.30) - open/close 4677 (time 1781231166.62 total 80.11 last 55.44) - open/close 5377 (time 1781231176.63 total 90.13 last 69.86) - open/close 6021 (time 1781231186.64 total 100.13 last 64.38) - open/close 6664 (time 1781231196.64 total 110.13 last 64.28) - open/close 7349 (time 1781231206.65 total 120.14 last 68.47) - open/close 7984 (time 1781231216.65 total 130.14 last 63.47) - open/close 8597 (time 1781231226.67 total 140.16 last 61.19) - open/close 9222 (time 1781231236.68 total 150.17 last 62.44) - open/close 9774 (time 1781231246.68 total 160.17 last 55.18) total: 10000 open/close in 165.09 seconds: 60.57 ops/second llite.lustre-ffff9f1f8478e000.statahead_batch_max=0 llite.lustre-ffff9f1f8478e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.758s user 0m0.289s sys 0m14.776s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f1f8478e000.statahead_batch_max=32 llite.lustre-ffff9f1f8478e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.611s user 0m0.309s sys 0m14.844s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats= snapshot_time: 1781231461.220342785 secs.nsecs start_time: 1781231337.461552369 secs.nsecs elapsed_time: 123.758790416 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=0 llite.lustre-ffff9f1f8478e000.statahead_max=2048 llite.lustre-ffff9f1f8478e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=0 total: 100 open/close in 4.27 seconds: 23.42 ops/second llite.lustre-ffff9f1f8478e000.statahead_max=0 101 real 0m1.624s user 0m0.001s sys 0m0.259s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9f1f8478e000.statahead_max=2048 llite.lustre-ffff9f1f8478e000.statahead_max=2048 101 real 0m0.802s user 0m0.007s sys 0m0.151s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 623 (time 1781231815.07 total 10.02 last 62.16) total: 900 open/close in 14.27 seconds: 63.06 ops/second llite.lustre-ffff9f1f8478e000.statahead_max=0 1001 real 0m32.574s user 0m0.041s sys 0m3.313s 'ls -l' 1000 files without statahead: 33 sec llite.lustre-ffff9f1f8478e000.statahead_max=2048 llite.lustre-ffff9f1f8478e000.statahead_max=2048 1001 real 0m8.926s user 0m0.040s sys 0m1.557s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9f1f8478e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats= snapshot_time: 1781231901.712295478 secs.nsecs start_time: 1781231464.049576793 secs.nsecs elapsed_time: 437.662718685 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 3 27 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 555 (time 1781231915.00 total 10.02 last 55.40) - open/close 1123 (time 1781231925.00 total 20.02 last 56.80) - open/close 1755 (time 1781231935.01 total 30.03 last 63.16) - open/close 2370 (time 1781231945.02 total 40.04 last 61.41) - open/close 3093 (time 1781231955.03 total 50.05 last 72.24) - open/close 3715 (time 1781231965.04 total 60.06 last 62.14) - open/close 4410 (time 1781231975.04 total 70.06 last 69.49) - open/close 5066 (time 1781231985.05 total 80.07 last 65.56) - open/close 5574 (time 1781231995.07 total 90.09 last 50.67) - open/close 6275 (time 1781232005.08 total 100.10 last 70.05) - open/close 6707 (time 1781232015.10 total 110.12 last 43.12) - open/close 7143 (time 1781232025.10 total 120.12 last 43.60) - open/close 7678 (time 1781232035.10 total 130.12 last 53.48) - open/close 8267 (time 1781232045.11 total 140.12 last 58.89) - open/close 8863 (time 1781232055.11 total 150.13 last 59.57) - open/close 9400 (time 1781232065.14 total 160.16 last 53.54) - open/close 9895 (time 1781232075.15 total 170.16 last 49.48) total: 10000 open/close in 171.85 seconds: 58.19 ops/second llite.lustre-ffff9f1f8478e000.statahead_batch_max=0 llite.lustre-ffff9f1f8478e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.894s user 0m0.305s sys 0m15.037s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f1f8478e000.statahead_batch_max=256 llite.lustre-ffff9f1f8478e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.993s user 0m0.302s sys 0m15.478s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f1f8478e000.batch_stats= snapshot_time: 1781232246.133099907 secs.nsecs start_time: 1781232167.176877282 secs.nsecs elapsed_time: 78.956222625 secs.nsecs subreqs per batch batches % cum % 1: 178 53 53 2: 40 11 65 4: 31 9 74 8: 21 6 80 16: 9 2 83 32: 13 3 87 64: 9 2 89 128: 0 0 89 256: 34 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f1f8478e000.statahead_batch_max=64 llite.lustre-ffff9f1f8478e000.statahead_max=128