== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:55:34 (1787860534) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=0 llite.lustre-ffff8d9846a0f000.statahead_max=32 llite.lustre-ffff8d9846a0f000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=0 total: 100 open/close in 3.26 seconds: 30.66 ops/second llite.lustre-ffff8d9846a0f000.statahead_max=0 101 real 0m2.863s user 0m0.006s sys 0m0.366s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8d9846a0f000.statahead_max=32 llite.lustre-ffff8d9846a0f000.statahead_max=32 101 real 0m1.236s user 0m0.006s sys 0m0.165s 'ls -l' 100 files with statahead: 2 sec statahead total: 101 statahead wrong: 2 agl total: 94 list_total: 30 fname_total: 70 hit_total: 4175 miss_total: 772 swrong=2 ewrong=2 - open/close 657 (time 1787860562.95 total 10.01 last 65.63) total: 900 open/close in 13.47 seconds: 66.81 ops/second llite.lustre-ffff8d9846a0f000.statahead_max=0 1001 real 0m23.331s user 0m0.035s sys 0m3.128s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8d9846a0f000.statahead_max=32 llite.lustre-ffff8d9846a0f000.statahead_max=32 1001 real 0m11.074s user 0m0.032s sys 0m1.461s 'ls -l' 1000 files with statahead: 11 sec statahead total: 102 statahead wrong: 2 agl total: 95 list_total: 31 fname_total: 70 hit_total: 5173 miss_total: 774 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8d9846a0f000.statahead_stats= statahead total: 102 statahead wrong: 2 agl total: 95 list_total: 31 fname_total: 70 hit_total: 5173 miss_total: 774 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats= snapshot_time: 1787860634.016416712 secs.nsecs start_time: 1787860539.525533276 secs.nsecs elapsed_time: 94.490883436 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 693 (time 1787860646.38 total 10.00 last 69.28) - open/close 1371 (time 1787860656.40 total 20.02 last 67.68) - open/close 1984 (time 1787860666.41 total 30.03 last 61.22) - open/close 2629 (time 1787860676.42 total 40.05 last 64.42) - open/close 3227 (time 1787860686.42 total 50.05 last 59.78) - open/close 3895 (time 1787860696.43 total 60.05 last 66.80) - open/close 4541 (time 1787860706.43 total 70.05 last 64.58) - open/close 5182 (time 1787860716.44 total 80.06 last 64.03) - open/close 5855 (time 1787860726.45 total 90.07 last 67.24) - open/close 6513 (time 1787860736.46 total 100.09 last 65.71) - open/close 7115 (time 1787860746.47 total 110.09 last 60.15) - open/close 7732 (time 1787860756.49 total 120.12 last 61.56) - open/close 8322 (time 1787860766.49 total 130.12 last 58.98) - open/close 8920 (time 1787860776.50 total 140.12 last 59.79) - open/close 9598 (time 1787860786.51 total 150.13 last 67.74) total: 10000 open/close in 156.19 seconds: 64.03 ops/second llite.lustre-ffff8d9846a0f000.statahead_batch_max=0 llite.lustre-ffff8d9846a0f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.550s user 0m0.408s sys 0m16.147s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d9846a0f000.statahead_batch_max=32 llite.lustre-ffff8d9846a0f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.277s user 0m0.340s sys 0m14.819s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats= snapshot_time: 1787860992.103015415 secs.nsecs start_time: 1787860879.805412785 secs.nsecs elapsed_time: 112.297602630 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=0 llite.lustre-ffff8d9846a0f000.statahead_max=2048 llite.lustre-ffff8d9846a0f000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=0 total: 100 open/close in 2.88 seconds: 34.78 ops/second llite.lustre-ffff8d9846a0f000.statahead_max=0 101 real 0m2.337s user 0m0.008s sys 0m0.306s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8d9846a0f000.statahead_max=2048 llite.lustre-ffff8d9846a0f000.statahead_max=2048 101 real 0m1.052s user 0m0.005s sys 0m0.156s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 657 (time 1787861323.23 total 10.01 last 65.67) total: 900 open/close in 14.13 seconds: 63.70 ops/second llite.lustre-ffff8d9846a0f000.statahead_max=0 1001 real 0m22.885s user 0m0.038s sys 0m2.864s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8d9846a0f000.statahead_max=2048 llite.lustre-ffff8d9846a0f000.statahead_max=2048 1001 real 0m7.670s user 0m0.026s sys 0m1.293s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8d9846a0f000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats= snapshot_time: 1787861392.384520543 secs.nsecs start_time: 1787860994.501615707 secs.nsecs elapsed_time: 397.882904836 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 0 0 36 64: 1 9 45 128: 3 27 72 256: 3 27 100 - open/close 690 (time 1787861404.86 total 10.01 last 68.91) - open/close 1434 (time 1787861414.88 total 20.02 last 74.32) - open/close 2125 (time 1787861424.88 total 30.03 last 69.04) - open/close 2792 (time 1787861434.90 total 40.04 last 66.62) - open/close 3429 (time 1787861444.90 total 50.05 last 63.69) - open/close 4088 (time 1787861454.90 total 60.05 last 65.86) - open/close 4704 (time 1787861464.92 total 70.07 last 61.50) - open/close 5444 (time 1787861474.93 total 80.07 last 73.95) - open/close 6145 (time 1787861484.94 total 90.08 last 70.03) - open/close 6807 (time 1787861494.94 total 100.09 last 66.17) - open/close 7434 (time 1787861504.95 total 110.10 last 62.65) - open/close 8140 (time 1787861514.95 total 120.10 last 70.60) - open/close 8792 (time 1787861524.96 total 130.11 last 65.13) - open/close 9446 (time 1787861534.97 total 140.11 last 65.36) total: 10000 open/close in 148.06 seconds: 67.54 ops/second llite.lustre-ffff8d9846a0f000.statahead_batch_max=0 llite.lustre-ffff8d9846a0f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.209s user 0m0.331s sys 0m15.500s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d9846a0f000.statahead_batch_max=256 llite.lustre-ffff8d9846a0f000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.056s user 0m0.291s sys 0m15.147s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8d9846a0f000.batch_stats= snapshot_time: 1787861672.577277988 secs.nsecs start_time: 1787861613.542365175 secs.nsecs elapsed_time: 59.034912813 secs.nsecs subreqs per batch batches %% cum %% 1: 128 48 48 2: 27 10 58 4: 31 11 70 8: 11 4 74 16: 10 3 78 32: 13 4 83 64: 7 2 85 128: 3 1 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8d9846a0f000.statahead_batch_max=64 llite.lustre-ffff8d9846a0f000.statahead_max=128