== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:08:15 (1777568895) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=0 llite.lustre-ffff9f0b4ad83000.statahead_max=32 llite.lustre-ffff9f0b4ad83000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.46 seconds: 68.63 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 101 real 0m1.590s user 0m0.007s sys 0m0.309s ls -l 100 files without statahead: 1 sec llite.lustre-ffff9f0b4ad83000.statahead_max=32 32 101 real 0m0.616s user 0m0.003s sys 0m0.195s ls -l 100 files with statahead: 1 sec statahead total: 54 statahead wrong: 1 agl total: 50 list_total: 26 fname_total: 28 hit_total: 33740 miss_total: 187 total: 900 open/close in 7.46 seconds: 120.65 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 1001 real 0m15.049s user 0m0.026s sys 0m2.978s ls -l 1000 files without statahead: 15 sec llite.lustre-ffff9f0b4ad83000.statahead_max=32 32 1001 real 0m5.035s user 0m0.025s sys 0m1.566s ls -l 1000 files with statahead: 5 sec statahead total: 55 statahead wrong: 1 agl total: 51 list_total: 27 fname_total: 28 hit_total: 34738 miss_total: 189 - open/close 1387 (time 1777568948.37 total 10.00 last 138.69) total: 2000 open/close in 14.00 seconds: 142.87 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 3001 real 0m35.810s user 0m0.098s sys 0m8.481s ls -l 3000 files without statahead: 36 sec llite.lustre-ffff9f0b4ad83000.statahead_max=32 32 3001 real 0m13.241s user 0m0.064s sys 0m4.792s ls -l 3000 files with statahead: 13 sec statahead total: 56 statahead wrong: 1 agl total: 52 list_total: 28 fname_total: 28 hit_total: 37734 miss_total: 193 ls -l done rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9f0b4ad83000.statahead_stats= statahead total: 56 statahead wrong: 1 agl total: 52 list_total: 28 fname_total: 28 hit_total: 37734 miss_total: 193 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569036.109981679 secs.nsecs start_time: 1777568898.434133132 secs.nsecs elapsed_time: 137.675848547 secs.nsecs subreqs per batch batches % cum % 1: 5 3 3 2: 1 0 4 4: 2 1 5 8: 3 2 8 16: 91 67 75 32: 33 24 100 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569036.110044742 secs.nsecs start_time: 1777568898.434387679 secs.nsecs elapsed_time: 137.675657063 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 0 0 4: 3 2 2 8: 3 2 5 16: 59 43 48 32: 69 51 100 - open/close 1988 (time 1777569047.56 total 10.00 last 198.77) - open/close 4247 (time 1777569057.56 total 20.00 last 225.84) - open/close 6289 (time 1777569067.57 total 30.01 last 204.16) - open/close 8338 (time 1777569077.57 total 40.01 last 204.90) total: 10000 open/close in 48.02 seconds: 208.24 ops/second llite.lustre-ffff9f0b4ad83000.statahead_batch_max=0 llite.lustre-ffff9f0b4ad83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.460s user 0m0.278s sys 0m27.935s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f0b4ad83000.statahead_batch_max=32 llite.lustre-ffff9f0b4ad83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.589s user 0m0.236s sys 0m20.847s batched statahead RPC= hit count=9994 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569170.297168465 secs.nsecs start_time: 1777569120.923209613 secs.nsecs elapsed_time: 49.373958852 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569170.297231685 secs.nsecs start_time: 1777569120.923365346 secs.nsecs elapsed_time: 49.373866339 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=0 llite.lustre-ffff9f0b4ad83000.statahead_max=2048 llite.lustre-ffff9f0b4ad83000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.36 seconds: 73.60 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 101 real 0m1.142s user 0m0.005s sys 0m0.303s ls -l 100 files without statahead: 1 sec llite.lustre-ffff9f0b4ad83000.statahead_max=2048 2048 101 real 0m0.510s user 0m0.007s sys 0m0.297s ls -l 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10093 miss_total: 7 total: 900 open/close in 4.76 seconds: 189.24 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 1001 real 0m10.639s user 0m0.033s sys 0m2.892s ls -l 1000 files without statahead: 11 sec llite.lustre-ffff9f0b4ad83000.statahead_max=2048 2048 1001 real 0m3.379s user 0m0.015s sys 0m1.638s ls -l 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11092 miss_total: 8 - open/close 1802 (time 1777569355.67 total 10.00 last 180.16) total: 3000 open/close in 17.04 seconds: 176.06 ops/second llite.lustre-ffff9f0b4ad83000.statahead_max=0 4001 real 0m38.624s user 0m0.135s sys 0m11.186s ls -l 4000 files without statahead: 39 sec llite.lustre-ffff9f0b4ad83000.statahead_max=2048 2048 4001 real 0m11.751s user 0m0.069s sys 0m7.267s ls -l 4000 files with statahead: 11 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15091 miss_total: 9 ls -l done rm -r /mnt/lustre/d123ad.sanity: 34 seconds llite.lustre-ffff9f0b4ad83000.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 15091 miss_total: 9 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569454.998391906 secs.nsecs start_time: 1777569171.393113956 secs.nsecs elapsed_time: 283.605277950 secs.nsecs subreqs per batch batches % cum % 1: 78 64 64 2: 22 18 82 4: 8 6 89 8: 1 0 90 16: 0 0 90 32: 0 0 90 64: 0 0 90 128: 3 2 92 256: 9 7 100 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569454.998458014 secs.nsecs start_time: 1777569171.393364026 secs.nsecs elapsed_time: 283.605093988 secs.nsecs subreqs per batch batches % cum % 1: 65 53 53 2: 25 20 74 4: 12 9 84 8: 3 2 86 16: 1 0 87 32: 0 0 87 64: 3 2 90 128: 1 0 90 256: 11 9 100 - open/close 2714 (time 1777569465.92 total 10.00 last 271.37) - open/close 5065 (time 1777569475.92 total 20.00 last 235.04) - open/close 6887 (time 1777569485.92 total 30.00 last 182.19) - open/close 9141 (time 1777569495.92 total 40.01 last 225.35) total: 10000 open/close in 42.92 seconds: 233.00 ops/second llite.lustre-ffff9f0b4ad83000.statahead_batch_max=0 llite.lustre-ffff9f0b4ad83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.875s user 0m0.206s sys 0m27.079s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f0b4ad83000.statahead_batch_max=256 llite.lustre-ffff9f0b4ad83000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.stats=clear mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m27.865s user 0m0.190s sys 0m20.534s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569568.966793323 secs.nsecs start_time: 1777569534.379492146 secs.nsecs elapsed_time: 34.587301177 secs.nsecs subreqs per batch batches % cum % 1: 418 63 63 2: 83 12 75 4: 70 10 86 8: 26 3 90 16: 8 1 91 32: 19 2 94 64: 4 0 95 128: 1 0 95 256: 32 4 100 mdc.lustre-MDT0001-mdc-ffff9f0b4ad83000.batch_stats= snapshot_time: 1777569568.966847473 secs.nsecs start_time: 1777569534.379567763 secs.nsecs elapsed_time: 34.587279710 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f0b4ad83000.statahead_batch_max=64 llite.lustre-ffff9f0b4ad83000.statahead_max=128