== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:43:41 (1787859821) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=0 llite.lustre-ffff988f122cd000.statahead_max=32 llite.lustre-ffff988f122cd000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=0 total: 100 open/close in 2.49 seconds: 40.14 ops/second llite.lustre-ffff988f122cd000.statahead_max=0 101 real 0m2.563s user 0m0.004s sys 0m0.302s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff988f122cd000.statahead_max=32 llite.lustre-ffff988f122cd000.statahead_max=32 101 real 0m1.981s user 0m0.007s sys 0m0.172s 'ls -l' 100 files with statahead: 2 sec statahead total: 118 statahead wrong: 2 agl total: 112 list_total: 30 fname_total: 74 hit_total: 4088 miss_total: 1595 swrong=2 ewrong=2 - open/close 470 (time 1787859852.98 total 10.00 last 46.99) - open/close 852 (time 1787859862.99 total 20.01 last 38.17) total: 900 open/close in 21.23 seconds: 42.39 ops/second llite.lustre-ffff988f122cd000.statahead_max=0 1001 real 0m27.426s user 0m0.037s sys 0m2.960s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff988f122cd000.statahead_max=32 llite.lustre-ffff988f122cd000.statahead_max=32 1001 real 0m12.338s user 0m0.030s sys 0m1.346s 'ls -l' 1000 files with statahead: 13 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 74 hit_total: 5086 miss_total: 1597 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff988f122cd000.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 31 fname_total: 74 hit_total: 5086 miss_total: 1597 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats= snapshot_time: 1787859951.425166878 secs.nsecs start_time: 1787859826.849169540 secs.nsecs elapsed_time: 124.575997338 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 517 (time 1787859965.00 total 10.00 last 51.68) - open/close 1031 (time 1787859975.01 total 20.02 last 51.32) - open/close 1664 (time 1787859985.02 total 30.03 last 63.23) - open/close 2250 (time 1787859995.04 total 40.04 last 58.53) - open/close 2831 (time 1787860005.07 total 50.07 last 57.92) - open/close 3371 (time 1787860015.09 total 60.09 last 53.89) - open/close 3934 (time 1787860025.10 total 70.11 last 56.22) - open/close 4566 (time 1787860035.11 total 80.12 last 63.14) - open/close 5189 (time 1787860045.13 total 90.14 last 62.17) - open/close 5646 (time 1787860055.14 total 100.14 last 45.68) - open/close 6251 (time 1787860065.15 total 110.16 last 60.43) - open/close 6864 (time 1787860075.15 total 120.16 last 61.28) - open/close 7540 (time 1787860085.16 total 130.17 last 67.55) - open/close 8323 (time 1787860095.17 total 140.17 last 78.23) - open/close 9102 (time 1787860105.18 total 150.19 last 77.81) - open/close 9810 (time 1787860115.18 total 160.19 last 70.80) total: 10000 open/close in 162.67 seconds: 61.47 ops/second llite.lustre-ffff988f122cd000.statahead_batch_max=0 llite.lustre-ffff988f122cd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m15.078s user 0m0.302s sys 0m13.626s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff988f122cd000.statahead_batch_max=32 llite.lustre-ffff988f122cd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m59.691s user 0m0.311s sys 0m13.421s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats= snapshot_time: 1787860336.452288016 secs.nsecs start_time: 1787860202.129798355 secs.nsecs elapsed_time: 134.322489661 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=0 llite.lustre-ffff988f122cd000.statahead_max=2048 llite.lustre-ffff988f122cd000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=0 total: 100 open/close in 4.12 seconds: 24.25 ops/second llite.lustre-ffff988f122cd000.statahead_max=0 101 real 0m3.250s user 0m0.003s sys 0m0.304s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff988f122cd000.statahead_max=2048 llite.lustre-ffff988f122cd000.statahead_max=2048 101 real 0m1.363s user 0m0.003s sys 0m0.194s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 689 (time 1787860685.42 total 10.01 last 68.82) total: 900 open/close in 14.30 seconds: 62.92 ops/second llite.lustre-ffff988f122cd000.statahead_max=0 1001 real 0m22.383s user 0m0.033s sys 0m3.053s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff988f122cd000.statahead_max=2048 llite.lustre-ffff988f122cd000.statahead_max=2048 1001 real 0m6.255s user 0m0.025s sys 0m1.370s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff988f122cd000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats= snapshot_time: 1787860760.606300751 secs.nsecs start_time: 1787860338.927234280 secs.nsecs elapsed_time: 421.679066471 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 1 9 45 64: 2 18 63 128: 0 0 63 256: 4 36 100 - open/close 534 (time 1787860773.17 total 10.00 last 53.39) - open/close 1060 (time 1787860783.20 total 20.03 last 52.47) - open/close 1654 (time 1787860793.24 total 30.07 last 59.12) - open/close 2196 (time 1787860803.27 total 40.10 last 54.07) - open/close 2845 (time 1787860813.27 total 50.10 last 64.86) - open/close 3453 (time 1787860823.28 total 60.11 last 60.73) - open/close 4071 (time 1787860833.28 total 70.11 last 61.80) - open/close 4745 (time 1787860843.29 total 80.12 last 67.38) - open/close 5380 (time 1787860853.29 total 90.12 last 63.49) - open/close 5959 (time 1787860863.30 total 100.13 last 57.85) - open/close 6642 (time 1787860873.30 total 110.13 last 68.27) - open/close 7291 (time 1787860883.31 total 120.14 last 64.82) - open/close 7906 (time 1787860893.33 total 130.16 last 61.42) - open/close 8592 (time 1787860903.35 total 140.17 last 68.48) - open/close 9175 (time 1787860913.35 total 150.18 last 58.28) - open/close 9855 (time 1787860923.36 total 160.19 last 67.90) total: 10000 open/close in 163.07 seconds: 61.32 ops/second llite.lustre-ffff988f122cd000.statahead_batch_max=0 llite.lustre-ffff988f122cd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.333s user 0m0.312s sys 0m13.651s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff988f122cd000.statahead_batch_max=256 llite.lustre-ffff988f122cd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff988f122cd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.183s user 0m0.317s sys 0m13.746s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff988f122cd000.batch_stats= snapshot_time: 1787861078.315660298 secs.nsecs start_time: 1787861004.873280068 secs.nsecs elapsed_time: 73.442380230 secs.nsecs subreqs per batch batches %% cum %% 1: 78 44 44 2: 12 6 51 4: 11 6 57 8: 6 3 61 16: 13 7 68 32: 16 9 77 64: 2 1 78 128: 1 0 79 256: 36 20 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff988f122cd000.statahead_batch_max=64 llite.lustre-ffff988f122cd000.statahead_max=128