== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:37:01 (1785969421) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=0 llite.lustre-ffff8d6a488c2000.statahead_max=32 llite.lustre-ffff8d6a488c2000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=0 total: 100 open/close in 2.68 seconds: 37.35 ops/second llite.lustre-ffff8d6a488c2000.statahead_max=0 101 real 0m1.980s user 0m0.006s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8d6a488c2000.statahead_max=32 llite.lustre-ffff8d6a488c2000.statahead_max=32 101 real 0m0.999s user 0m0.003s sys 0m0.181s 'ls -l' 100 files with statahead: 1 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 80 hit_total: 3651 miss_total: 1645 swrong=2 ewrong=2 - open/close 708 (time 1785969449.78 total 10.01 last 70.74) total: 900 open/close in 12.35 seconds: 72.90 ops/second llite.lustre-ffff8d6a488c2000.statahead_max=0 1001 real 0m18.348s user 0m0.040s sys 0m3.137s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff8d6a488c2000.statahead_max=32 llite.lustre-ffff8d6a488c2000.statahead_max=32 1001 real 0m10.471s user 0m0.032s sys 0m1.575s 'ls -l' 1000 files with statahead: 11 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 32 fname_total: 80 hit_total: 4650 miss_total: 1646 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8d6a488c2000.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 32 fname_total: 80 hit_total: 4650 miss_total: 1646 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats= snapshot_time: 1785969515.354090115 secs.nsecs start_time: 1785969426.526081784 secs.nsecs elapsed_time: 88.828008331 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 655 (time 1785969527.99 total 10.01 last 65.46) - open/close 1477 (time 1785969538.00 total 20.01 last 82.13) - open/close 2244 (time 1785969548.02 total 30.03 last 76.58) - open/close 3106 (time 1785969558.02 total 40.03 last 86.18) - open/close 4016 (time 1785969568.03 total 50.04 last 90.93) - open/close 4894 (time 1785969578.04 total 60.05 last 87.71) - open/close 5618 (time 1785969588.04 total 70.06 last 72.36) - open/close 6416 (time 1785969598.05 total 80.06 last 79.75) - open/close 7105 (time 1785969608.05 total 90.06 last 68.89) - open/close 7721 (time 1785969618.06 total 100.07 last 61.59) - open/close 8229 (time 1785969628.07 total 110.08 last 50.73) - open/close 8864 (time 1785969638.12 total 120.13 last 63.18) - open/close 9515 (time 1785969648.12 total 130.13 last 65.08) total: 10000 open/close in 136.93 seconds: 73.03 ops/second llite.lustre-ffff8d6a488c2000.statahead_batch_max=0 llite.lustre-ffff8d6a488c2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.449s user 0m0.302s sys 0m15.015s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d6a488c2000.statahead_batch_max=32 llite.lustre-ffff8d6a488c2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.847s user 0m0.309s sys 0m14.713s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats= snapshot_time: 1785969853.021118020 secs.nsecs start_time: 1785969729.859199581 secs.nsecs elapsed_time: 123.161918439 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=0 llite.lustre-ffff8d6a488c2000.statahead_max=2048 llite.lustre-ffff8d6a488c2000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=0 total: 100 open/close in 2.46 seconds: 40.64 ops/second llite.lustre-ffff8d6a488c2000.statahead_max=0 101 real 0m2.115s user 0m0.005s sys 0m0.327s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8d6a488c2000.statahead_max=2048 llite.lustre-ffff8d6a488c2000.statahead_max=2048 101 real 0m0.689s user 0m0.003s sys 0m0.153s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 - open/close 885 (time 1785970137.36 total 10.00 last 88.49) total: 900 open/close in 10.14 seconds: 88.78 ops/second llite.lustre-ffff8d6a488c2000.statahead_max=0 1001 real 0m20.121s user 0m0.035s sys 0m3.174s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8d6a488c2000.statahead_max=2048 llite.lustre-ffff8d6a488c2000.statahead_max=2048 1001 real 0m7.761s user 0m0.030s sys 0m1.390s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff8d6a488c2000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats= snapshot_time: 1785970196.410322205 secs.nsecs start_time: 1785969855.542647588 secs.nsecs elapsed_time: 340.867674617 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 835 (time 1785970208.61 total 10.00 last 83.46) - open/close 1621 (time 1785970218.61 total 20.00 last 78.60) - open/close 2355 (time 1785970228.61 total 30.01 last 73.38) - open/close 3065 (time 1785970238.62 total 40.01 last 70.95) - open/close 3845 (time 1785970248.63 total 50.02 last 77.94) - open/close 4620 (time 1785970258.63 total 60.03 last 77.47) - open/close 5451 (time 1785970268.64 total 70.03 last 83.03) - open/close 6391 (time 1785970278.64 total 80.04 last 93.97) - open/close 7138 (time 1785970288.66 total 90.05 last 74.59) - open/close 7949 (time 1785970298.67 total 100.06 last 81.02) - open/close 8803 (time 1785970308.67 total 110.07 last 85.37) - open/close 9704 (time 1785970318.67 total 120.07 last 90.08) total: 10000 open/close in 123.14 seconds: 81.21 ops/second llite.lustre-ffff8d6a488c2000.statahead_batch_max=0 llite.lustre-ffff8d6a488c2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.556s user 0m0.313s sys 0m15.212s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d6a488c2000.statahead_batch_max=256 llite.lustre-ffff8d6a488c2000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.319s user 0m0.252s sys 0m15.766s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8d6a488c2000.batch_stats= snapshot_time: 1785970450.057573502 secs.nsecs start_time: 1785970387.839151755 secs.nsecs elapsed_time: 62.218421747 secs.nsecs subreqs per batch batches %% cum %% 1: 185 45 45 2: 60 14 60 4: 68 16 77 8: 25 6 83 16: 17 4 87 32: 9 2 89 64: 8 1 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8d6a488c2000.statahead_batch_max=64 llite.lustre-ffff8d6a488c2000.statahead_max=128