== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:37:32 (1787585852) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=0 llite.lustre-ffff9d1d3e372000.statahead_max=32 llite.lustre-ffff9d1d3e372000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=0 total: 100 open/close in 2.83 seconds: 35.31 ops/second llite.lustre-ffff9d1d3e372000.statahead_max=0 101 real 0m2.178s user 0m0.006s sys 0m0.294s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d1d3e372000.statahead_max=32 llite.lustre-ffff9d1d3e372000.statahead_max=32 101 real 0m1.153s user 0m0.004s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 29 fname_total: 97 hit_total: 4027 miss_total: 1574 swrong=2 ewrong=2 - open/close 747 (time 1787585879.65 total 10.01 last 74.65) total: 900 open/close in 12.06 seconds: 74.60 ops/second llite.lustre-ffff9d1d3e372000.statahead_max=0 1001 real 0m20.894s user 0m0.030s sys 0m2.912s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9d1d3e372000.statahead_max=32 llite.lustre-ffff9d1d3e372000.statahead_max=32 1001 real 0m8.649s user 0m0.028s sys 0m1.247s 'ls -l' 1000 files with statahead: 8 sec statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 30 fname_total: 97 hit_total: 5026 miss_total: 1575 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9d1d3e372000.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 30 fname_total: 97 hit_total: 5026 miss_total: 1575 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats= snapshot_time: 1787585946.047201353 secs.nsecs start_time: 1787585856.993395227 secs.nsecs elapsed_time: 89.053806126 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 1 2 7 4: 0 0 7 8: 2 4 12 16: 2 4 17 32: 34 82 100 - open/close 658 (time 1787585959.28 total 10.00 last 65.80) - open/close 1490 (time 1787585969.30 total 20.01 last 83.10) - open/close 2257 (time 1787585979.30 total 30.01 last 76.69) - open/close 2999 (time 1787585989.30 total 40.02 last 74.15) - open/close 3562 (time 1787585999.32 total 50.03 last 56.23) - open/close 4324 (time 1787586009.32 total 60.03 last 76.20) - open/close 5050 (time 1787586019.33 total 70.04 last 72.52) - open/close 5725 (time 1787586029.33 total 80.05 last 67.46) - open/close 6494 (time 1787586039.33 total 90.05 last 76.90) - open/close 7342 (time 1787586049.34 total 100.06 last 84.71) - open/close 8096 (time 1787586059.35 total 110.06 last 75.38) - open/close 8857 (time 1787586069.35 total 120.06 last 76.09) - open/close 9523 (time 1787586079.37 total 130.08 last 66.46) total: 10000 open/close in 137.21 seconds: 72.88 ops/second llite.lustre-ffff9d1d3e372000.statahead_batch_max=0 llite.lustre-ffff9d1d3e372000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.091s user 0m0.332s sys 0m14.119s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d1d3e372000.statahead_batch_max=32 llite.lustre-ffff9d1d3e372000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.741s user 0m0.302s sys 0m13.005s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats= snapshot_time: 1787586282.013616083 secs.nsecs start_time: 1787586163.721755675 secs.nsecs elapsed_time: 118.291860408 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=0 llite.lustre-ffff9d1d3e372000.statahead_max=2048 llite.lustre-ffff9d1d3e372000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=0 total: 100 open/close in 2.89 seconds: 34.61 ops/second llite.lustre-ffff9d1d3e372000.statahead_max=0 101 real 0m3.282s user 0m0.012s sys 0m0.362s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9d1d3e372000.statahead_max=2048 llite.lustre-ffff9d1d3e372000.statahead_max=2048 101 real 0m0.681s user 0m0.004s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 722 (time 1787586596.05 total 10.00 last 72.18) total: 900 open/close in 11.97 seconds: 75.19 ops/second llite.lustre-ffff9d1d3e372000.statahead_max=0 1001 real 0m19.008s user 0m0.043s sys 0m2.662s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9d1d3e372000.statahead_max=2048 llite.lustre-ffff9d1d3e372000.statahead_max=2048 1001 real 0m5.744s user 0m0.028s sys 0m1.336s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff9d1d3e372000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats= snapshot_time: 1787586651.865647596 secs.nsecs start_time: 1787586285.430146576 secs.nsecs elapsed_time: 366.435501020 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 15 15 16: 2 15 30 32: 3 23 53 64: 2 15 69 128: 0 0 69 256: 4 30 100 - open/close 835 (time 1787586664.44 total 10.00 last 83.47) - open/close 1632 (time 1787586674.45 total 20.01 last 79.66) - open/close 2424 (time 1787586684.47 total 30.03 last 79.02) - open/close 3252 (time 1787586694.47 total 40.03 last 82.78) - open/close 4005 (time 1787586704.48 total 50.04 last 75.25) - open/close 4847 (time 1787586714.49 total 60.05 last 84.15) - open/close 5540 (time 1787586724.51 total 70.07 last 69.15) - open/close 6363 (time 1787586734.51 total 80.07 last 82.25) - open/close 7174 (time 1787586744.51 total 90.08 last 81.09) - open/close 7878 (time 1787586754.52 total 100.08 last 70.34) - open/close 8734 (time 1787586764.53 total 110.10 last 85.50) - open/close 9587 (time 1787586774.55 total 120.11 last 85.20) total: 10000 open/close in 125.50 seconds: 79.68 ops/second llite.lustre-ffff9d1d3e372000.statahead_batch_max=0 llite.lustre-ffff9d1d3e372000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.652s user 0m0.283s sys 0m13.546s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d1d3e372000.statahead_batch_max=256 llite.lustre-ffff9d1d3e372000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.512s user 0m0.265s sys 0m12.889s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d1d3e372000.batch_stats= snapshot_time: 1787586898.837537436 secs.nsecs start_time: 1787586843.665307603 secs.nsecs elapsed_time: 55.172229833 secs.nsecs subreqs per batch batches %% cum %% 1: 62 35 35 2: 19 10 46 4: 20 11 57 8: 8 4 61 16: 12 6 68 32: 14 7 76 64: 5 2 79 128: 1 0 80 256: 35 19 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d1d3e372000.statahead_batch_max=64 llite.lustre-ffff9d1d3e372000.statahead_max=128