== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:50:40 (1789055440) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=0 llite.lustre-ffff95a12e816800.statahead_max=32 llite.lustre-ffff95a12e816800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=0 total: 100 open/close in 3.84 seconds: 26.04 ops/second llite.lustre-ffff95a12e816800.statahead_max=0 101 real 0m2.663s user 0m0.009s sys 0m0.388s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95a12e816800.statahead_max=32 llite.lustre-ffff95a12e816800.statahead_max=32 101 real 0m1.065s user 0m0.003s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 130 statahead wrong: 2 agl total: 125 list_total: 31 fname_total: 97 hit_total: 4448 miss_total: 1292 swrong=2 ewrong=2 - open/close 521 (time 1789055472.09 total 10.00 last 52.08) total: 900 open/close in 17.17 seconds: 52.43 ops/second llite.lustre-ffff95a12e816800.statahead_max=0 1001 real 0m27.697s user 0m0.035s sys 0m3.252s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff95a12e816800.statahead_max=32 llite.lustre-ffff95a12e816800.statahead_max=32 1001 real 0m12.737s user 0m0.036s sys 0m1.562s 'ls -l' 1000 files with statahead: 13 sec statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 32 fname_total: 97 hit_total: 5445 miss_total: 1295 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff95a12e816800.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 32 fname_total: 97 hit_total: 5445 miss_total: 1295 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats= snapshot_time: 1789055565.625211707 secs.nsecs start_time: 1789055446.583197437 secs.nsecs elapsed_time: 119.042014270 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 651 (time 1789055578.99 total 10.01 last 65.03) - open/close 1293 (time 1789055589.00 total 20.02 last 64.14) - open/close 1832 (time 1789055599.01 total 30.03 last 53.84) - open/close 2396 (time 1789055609.04 total 40.06 last 56.24) - open/close 3002 (time 1789055619.05 total 50.07 last 60.55) - open/close 3640 (time 1789055629.06 total 60.07 last 63.76) - open/close 4302 (time 1789055639.07 total 70.08 last 66.13) - open/close 4914 (time 1789055649.07 total 80.09 last 61.18) - open/close 5465 (time 1789055659.08 total 90.09 last 55.06) - open/close 6050 (time 1789055669.09 total 100.11 last 58.43) - open/close 6650 (time 1789055679.10 total 110.12 last 59.90) - open/close 7215 (time 1789055689.10 total 120.12 last 56.49) - open/close 7814 (time 1789055699.12 total 130.14 last 59.82) - open/close 8483 (time 1789055709.15 total 140.17 last 66.70) - open/close 9070 (time 1789055719.15 total 150.17 last 58.66) - open/close 9543 (time 1789055729.16 total 160.18 last 47.29) total: 10000 open/close in 167.26 seconds: 59.79 ops/second llite.lustre-ffff95a12e816800.statahead_batch_max=0 llite.lustre-ffff95a12e816800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m14.162s user 0m0.335s sys 0m15.917s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95a12e816800.statahead_batch_max=32 llite.lustre-ffff95a12e816800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.176s user 0m0.338s sys 0m14.794s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats= snapshot_time: 1789055938.198413851 secs.nsecs start_time: 1789055821.650529421 secs.nsecs elapsed_time: 116.547884430 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=0 llite.lustre-ffff95a12e816800.statahead_max=2048 llite.lustre-ffff95a12e816800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=0 total: 100 open/close in 3.38 seconds: 29.55 ops/second llite.lustre-ffff95a12e816800.statahead_max=0 101 real 0m2.830s user 0m0.004s sys 0m0.352s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95a12e816800.statahead_max=2048 llite.lustre-ffff95a12e816800.statahead_max=2048 101 real 0m1.242s user 0m0.007s sys 0m0.178s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 633 (time 1789056330.61 total 10.00 last 63.28) total: 900 open/close in 15.05 seconds: 59.79 ops/second llite.lustre-ffff95a12e816800.statahead_max=0 1001 real 0m30.655s user 0m0.041s sys 0m3.253s 'ls -l' 1000 files without statahead: 31 sec llite.lustre-ffff95a12e816800.statahead_max=2048 llite.lustre-ffff95a12e816800.statahead_max=2048 1001 real 0m5.059s user 0m0.023s sys 0m1.527s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff95a12e816800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats= snapshot_time: 1789056413.923048006 secs.nsecs start_time: 1789055942.501429908 secs.nsecs elapsed_time: 471.421618098 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 560 (time 1789056427.19 total 10.00 last 56.00) - open/close 1160 (time 1789056437.20 total 20.01 last 59.95) - open/close 1767 (time 1789056447.21 total 30.02 last 60.65) - open/close 2364 (time 1789056457.22 total 40.03 last 59.63) - open/close 2951 (time 1789056467.22 total 50.03 last 58.69) - open/close 3575 (time 1789056477.22 total 60.03 last 62.39) - open/close 4173 (time 1789056487.23 total 70.04 last 59.75) - open/close 4817 (time 1789056497.23 total 80.04 last 64.39) - open/close 5424 (time 1789056507.25 total 90.05 last 60.63) - open/close 6021 (time 1789056517.25 total 100.06 last 59.69) - open/close 6584 (time 1789056527.25 total 110.06 last 56.28) - open/close 7219 (time 1789056537.26 total 120.06 last 63.46) - open/close 7860 (time 1789056547.26 total 130.07 last 64.07) - open/close 8472 (time 1789056557.27 total 140.08 last 61.14) - open/close 9093 (time 1789056567.27 total 150.08 last 62.09) - open/close 9589 (time 1789056577.28 total 160.09 last 49.55) total: 10000 open/close in 167.42 seconds: 59.73 ops/second llite.lustre-ffff95a12e816800.statahead_batch_max=0 llite.lustre-ffff95a12e816800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m31.344s user 0m0.419s sys 0m16.624s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95a12e816800.statahead_batch_max=256 llite.lustre-ffff95a12e816800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95a12e816800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.921s user 0m0.372s sys 0m16.176s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95a12e816800.batch_stats= snapshot_time: 1789056760.507555400 secs.nsecs start_time: 1789056687.350301652 secs.nsecs elapsed_time: 73.157253748 secs.nsecs subreqs per batch batches %% cum %% 1: 124 46 46 2: 30 11 58 4: 25 9 67 8: 18 6 74 16: 12 4 78 32: 18 6 85 64: 3 1 86 128: 0 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95a12e816800.statahead_batch_max=64 llite.lustre-ffff95a12e816800.statahead_max=128