== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:54:59 (1789070099) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=0 llite.lustre-ffff9f98ca66c800.statahead_max=32 llite.lustre-ffff9f98ca66c800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=0 total: 100 open/close in 3.36 seconds: 29.73 ops/second llite.lustre-ffff9f98ca66c800.statahead_max=0 101 real 0m2.929s user 0m0.004s sys 0m0.332s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f98ca66c800.statahead_max=32 llite.lustre-ffff9f98ca66c800.statahead_max=32 101 real 0m1.120s user 0m0.003s sys 0m0.148s 'ls -l' 100 files with statahead: 1 sec statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 30 fname_total: 95 hit_total: 4119 miss_total: 1756 swrong=2 ewrong=2 - open/close 578 (time 1789070129.05 total 10.00 last 57.77) total: 900 open/close in 15.52 seconds: 57.99 ops/second llite.lustre-ffff9f98ca66c800.statahead_max=0 1001 real 0m24.766s user 0m0.049s sys 0m3.374s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9f98ca66c800.statahead_max=32 llite.lustre-ffff9f98ca66c800.statahead_max=32 1001 real 0m11.385s user 0m0.025s sys 0m1.576s 'ls -l' 1000 files with statahead: 11 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 95 hit_total: 5115 miss_total: 1760 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff9f98ca66c800.statahead_stats= statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 95 hit_total: 5115 miss_total: 1760 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats= snapshot_time: 1789070211.580056413 secs.nsecs start_time: 1789070104.621652517 secs.nsecs elapsed_time: 106.958403896 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 563 (time 1789070224.55 total 10.01 last 56.25) - open/close 1154 (time 1789070234.56 total 20.02 last 59.05) - open/close 1817 (time 1789070244.57 total 30.03 last 66.21) - open/close 2437 (time 1789070254.58 total 40.04 last 61.93) - open/close 3026 (time 1789070264.59 total 50.05 last 58.85) - open/close 3593 (time 1789070274.62 total 60.08 last 56.54) - open/close 4082 (time 1789070284.65 total 70.11 last 48.76) - open/close 4570 (time 1789070294.67 total 80.13 last 48.71) - open/close 5158 (time 1789070304.68 total 90.14 last 58.73) - open/close 5718 (time 1789070314.68 total 100.14 last 55.98) - open/close 6312 (time 1789070324.70 total 110.16 last 59.32) - open/close 6896 (time 1789070334.71 total 120.16 last 58.35) - open/close 7464 (time 1789070344.71 total 130.17 last 56.78) - open/close 8119 (time 1789070354.72 total 140.18 last 65.42) - open/close 8771 (time 1789070364.74 total 150.20 last 65.08) - open/close 9399 (time 1789070374.75 total 160.20 last 62.77) - open/close 9960 (time 1789070384.75 total 170.21 last 56.05) total: 10000 open/close in 171.01 seconds: 58.48 ops/second llite.lustre-ffff9f98ca66c800.statahead_batch_max=0 llite.lustre-ffff9f98ca66c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.750s user 0m0.341s sys 0m14.804s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f98ca66c800.statahead_batch_max=32 llite.lustre-ffff9f98ca66c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m1.126s user 0m0.303s sys 0m14.313s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats= snapshot_time: 1789070599.818508664 secs.nsecs start_time: 1789070466.391049161 secs.nsecs elapsed_time: 133.427459503 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=0 llite.lustre-ffff9f98ca66c800.statahead_max=2048 llite.lustre-ffff9f98ca66c800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.39 ops/second llite.lustre-ffff9f98ca66c800.statahead_max=0 101 real 0m2.819s user 0m0.004s sys 0m0.322s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f98ca66c800.statahead_max=2048 llite.lustre-ffff9f98ca66c800.statahead_max=2048 101 real 0m1.183s user 0m0.005s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 490 (time 1789070992.11 total 10.00 last 49.00) total: 900 open/close in 17.47 seconds: 51.53 ops/second llite.lustre-ffff9f98ca66c800.statahead_max=0 1001 real 0m30.341s user 0m0.038s sys 0m3.118s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff9f98ca66c800.statahead_max=2048 llite.lustre-ffff9f98ca66c800.statahead_max=2048 1001 real 0m7.862s user 0m0.025s sys 0m1.442s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff9f98ca66c800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats= snapshot_time: 1789071077.925677128 secs.nsecs start_time: 1789070602.507505064 secs.nsecs elapsed_time: 475.418172064 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 535 (time 1789071091.34 total 10.01 last 53.43) - open/close 1014 (time 1789071101.35 total 20.02 last 47.87) - open/close 1572 (time 1789071111.35 total 30.02 last 55.79) - open/close 2121 (time 1789071121.38 total 40.04 last 54.78) - open/close 2646 (time 1789071131.38 total 50.05 last 52.47) - open/close 3093 (time 1789071141.39 total 60.06 last 44.66) - open/close 3565 (time 1789071151.39 total 70.06 last 47.19) - open/close 4129 (time 1789071161.39 total 80.06 last 56.39) - open/close 4697 (time 1789071171.40 total 90.07 last 56.74) - open/close 5255 (time 1789071181.41 total 100.08 last 55.79) - open/close 5869 (time 1789071191.42 total 110.09 last 61.32) - open/close 6486 (time 1789071201.42 total 120.09 last 61.69) - open/close 7083 (time 1789071211.45 total 130.12 last 59.53) - open/close 7697 (time 1789071221.46 total 140.13 last 61.33) - open/close 8310 (time 1789071231.46 total 150.13 last 61.30) - open/close 8942 (time 1789071241.47 total 160.14 last 63.17) - open/close 9583 (time 1789071251.47 total 170.14 last 64.10) total: 10000 open/close in 178.74 seconds: 55.95 ops/second llite.lustre-ffff9f98ca66c800.statahead_batch_max=0 llite.lustre-ffff9f98ca66c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.901s user 0m0.365s sys 0m14.368s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f98ca66c800.statahead_batch_max=256 llite.lustre-ffff9f98ca66c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.973s user 0m0.315s sys 0m15.141s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f98ca66c800.batch_stats= snapshot_time: 1789071413.052259788 secs.nsecs start_time: 1789071343.418545618 secs.nsecs elapsed_time: 69.633714170 secs.nsecs subreqs per batch batches %% cum %% 1: 107 45 45 2: 27 11 57 4: 28 11 68 8: 9 3 72 16: 10 4 77 32: 11 4 81 64: 5 2 83 128: 3 1 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f98ca66c800.statahead_batch_max=64 llite.lustre-ffff9f98ca66c800.statahead_max=128