== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:57:13 (1788469033) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=0 llite.lustre-ffff9d289f3b0800.statahead_max=32 llite.lustre-ffff9d289f3b0800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=0 total: 100 open/close in 3.50 seconds: 28.59 ops/second llite.lustre-ffff9d289f3b0800.statahead_max=0 101 real 0m1.743s user 0m0.008s sys 0m0.284s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d289f3b0800.statahead_max=32 llite.lustre-ffff9d289f3b0800.statahead_max=32 101 real 0m1.611s user 0m0.007s sys 0m0.176s 'ls -l' 100 files with statahead: 2 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 30 fname_total: 76 hit_total: 4088 miss_total: 1564 swrong=2 ewrong=2 - open/close 500 (time 1788469065.34 total 10.00 last 49.99) total: 900 open/close in 17.48 seconds: 51.50 ops/second llite.lustre-ffff9d289f3b0800.statahead_max=0 1001 real 0m22.140s user 0m0.025s sys 0m2.828s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9d289f3b0800.statahead_max=32 llite.lustre-ffff9d289f3b0800.statahead_max=32 1001 real 0m11.702s user 0m0.030s sys 0m1.347s 'ls -l' 1000 files with statahead: 11 sec statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 31 fname_total: 76 hit_total: 5087 miss_total: 1565 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9d289f3b0800.statahead_stats= statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 31 fname_total: 76 hit_total: 5087 miss_total: 1565 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats= snapshot_time: 1788469144.430364807 secs.nsecs start_time: 1788469039.724852626 secs.nsecs elapsed_time: 104.705512181 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 665 (time 1788469157.11 total 10.03 last 66.33) - open/close 1313 (time 1788469167.16 total 20.07 last 64.51) - open/close 1777 (time 1788469177.16 total 30.07 last 46.40) - open/close 2263 (time 1788469187.17 total 40.08 last 48.55) - open/close 2938 (time 1788469197.18 total 50.09 last 67.44) - open/close 3798 (time 1788469207.18 total 60.09 last 86.00) - open/close 4552 (time 1788469217.21 total 70.12 last 75.17) - open/close 5200 (time 1788469227.21 total 80.13 last 64.79) - open/close 5765 (time 1788469237.24 total 90.15 last 56.34) - open/close 6225 (time 1788469247.24 total 100.16 last 45.97) - open/close 6744 (time 1788469257.25 total 110.17 last 51.86) - open/close 7404 (time 1788469267.25 total 120.17 last 65.98) - open/close 8065 (time 1788469277.26 total 130.18 last 66.05) - open/close 8657 (time 1788469287.27 total 140.18 last 59.17) - open/close 9248 (time 1788469297.28 total 150.20 last 59.02) - open/close 9859 (time 1788469307.29 total 160.20 last 61.06) total: 10000 open/close in 161.95 seconds: 61.75 ops/second llite.lustre-ffff9d289f3b0800.statahead_batch_max=0 llite.lustre-ffff9d289f3b0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.628s user 0m0.326s sys 0m13.919s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d289f3b0800.statahead_batch_max=32 llite.lustre-ffff9d289f3b0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.477s user 0m0.277s sys 0m12.661s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats= snapshot_time: 1788469488.196006028 secs.nsecs start_time: 1788469378.074920129 secs.nsecs elapsed_time: 110.121085899 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=0 llite.lustre-ffff9d289f3b0800.statahead_max=2048 llite.lustre-ffff9d289f3b0800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=0 total: 100 open/close in 3.14 seconds: 31.82 ops/second llite.lustre-ffff9d289f3b0800.statahead_max=0 101 real 0m1.991s user 0m0.005s sys 0m0.263s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d289f3b0800.statahead_max=2048 llite.lustre-ffff9d289f3b0800.statahead_max=2048 101 real 0m1.165s user 0m0.007s sys 0m0.149s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 699 (time 1788469827.58 total 10.01 last 69.83) total: 900 open/close in 12.52 seconds: 71.87 ops/second llite.lustre-ffff9d289f3b0800.statahead_max=0 1001 real 0m19.632s user 0m0.030s sys 0m2.891s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9d289f3b0800.statahead_max=2048 llite.lustre-ffff9d289f3b0800.statahead_max=2048 1001 real 0m6.178s user 0m0.030s sys 0m1.388s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9d289f3b0800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats= snapshot_time: 1788469897.280683729 secs.nsecs start_time: 1788469491.313868952 secs.nsecs elapsed_time: 405.966814777 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 750 (time 1788469911.06 total 10.05 last 74.63) - open/close 1276 (time 1788469921.07 total 20.06 last 52.55) - open/close 1998 (time 1788469931.07 total 30.07 last 72.14) - open/close 2782 (time 1788469941.08 total 40.07 last 78.35) - open/close 3492 (time 1788469951.08 total 50.08 last 70.97) - open/close 4035 (time 1788469961.09 total 60.09 last 54.25) - open/close 4567 (time 1788469971.09 total 70.09 last 53.20) - open/close 5053 (time 1788469981.10 total 80.09 last 48.56) - open/close 5842 (time 1788469991.12 total 90.11 last 78.74) - open/close 6608 (time 1788470001.13 total 100.12 last 76.53) - open/close 7264 (time 1788470011.14 total 110.13 last 65.56) - open/close 7865 (time 1788470021.16 total 120.16 last 59.94) - open/close 8408 (time 1788470031.16 total 130.16 last 54.29) - open/close 9028 (time 1788470041.19 total 140.18 last 61.84) - open/close 9668 (time 1788470051.19 total 150.19 last 63.98) total: 10000 open/close in 154.78 seconds: 64.61 ops/second llite.lustre-ffff9d289f3b0800.statahead_batch_max=0 llite.lustre-ffff9d289f3b0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.904s user 0m0.320s sys 0m13.642s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d289f3b0800.statahead_batch_max=256 llite.lustre-ffff9d289f3b0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.231s user 0m0.278s sys 0m13.525s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d289f3b0800.batch_stats= snapshot_time: 1788470209.630024586 secs.nsecs start_time: 1788470138.187762042 secs.nsecs elapsed_time: 71.442262544 secs.nsecs subreqs per batch batches %% cum %% 1: 67 36 36 2: 19 10 46 4: 22 11 58 8: 7 3 62 16: 12 6 69 32: 11 5 75 64: 9 4 79 128: 3 1 81 256: 34 18 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d289f3b0800.statahead_batch_max=64 llite.lustre-ffff9d289f3b0800.statahead_max=128