== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:21:32 (1789327292) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=0 llite.lustre-ffff96825a65c800.statahead_max=32 llite.lustre-ffff96825a65c800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.06 seconds: 94.59 ops/second llite.lustre-ffff96825a65c800.statahead_max=0 101 real 0m1.130s user 0m0.005s sys 0m0.359s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff96825a65c800.statahead_max=32 llite.lustre-ffff96825a65c800.statahead_max=32 101 real 0m0.618s user 0m0.006s sys 0m0.384s 'ls -l' 100 files with statahead: 0 sec statahead total: 86 statahead wrong: 2 agl total: 82 list_total: 28 fname_total: 58 hit_total: 3983 miss_total: 579 swrong=2 ewrong=2 total: 900 open/close in 3.94 seconds: 228.29 ops/second llite.lustre-ffff96825a65c800.statahead_max=0 1001 real 0m10.339s user 0m0.038s sys 0m3.345s 'ls -l' 1000 files without statahead: 10 sec llite.lustre-ffff96825a65c800.statahead_max=32 llite.lustre-ffff96825a65c800.statahead_max=32 1001 real 0m4.980s user 0m0.010s sys 0m1.821s 'ls -l' 1000 files with statahead: 5 sec statahead total: 87 statahead wrong: 2 agl total: 83 list_total: 29 fname_total: 58 hit_total: 4982 miss_total: 580 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff96825a65c800.statahead_stats= statahead total: 87 statahead wrong: 2 agl total: 83 list_total: 29 fname_total: 58 hit_total: 4982 miss_total: 580 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327335.591602613 secs.nsecs start_time: 1789327294.927004592 secs.nsecs elapsed_time: 40.664598021 secs.nsecs subreqs per batch batches %% cum %% 1: 4 10 10 2: 1 2 13 4: 0 0 13 8: 5 13 26 16: 21 55 81 32: 7 18 100 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327335.591737260 secs.nsecs start_time: 1789327294.927117217 secs.nsecs elapsed_time: 40.664620043 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 1 2 10 16: 14 35 46 32: 21 53 100 - open/close 2133 (time 1789327347.03 total 10.00 last 213.22) - open/close 4274 (time 1789327357.04 total 20.01 last 214.02) - open/close 6397 (time 1789327367.04 total 30.01 last 212.20) - open/close 8432 (time 1789327377.04 total 40.01 last 203.50) total: 10000 open/close in 47.52 seconds: 210.44 ops/second llite.lustre-ffff96825a65c800.statahead_batch_max=0 llite.lustre-ffff96825a65c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.469s user 0m0.148s sys 0m36.589s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96825a65c800.statahead_batch_max=32 llite.lustre-ffff96825a65c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.358s user 0m0.152s sys 0m33.597s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327491.420586594 secs.nsecs start_time: 1789327427.520470504 secs.nsecs elapsed_time: 63.900116090 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327491.420667974 secs.nsecs start_time: 1789327427.520580831 secs.nsecs elapsed_time: 63.900087143 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=0 llite.lustre-ffff96825a65c800.statahead_max=2048 llite.lustre-ffff96825a65c800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.42 seconds: 70.25 ops/second llite.lustre-ffff96825a65c800.statahead_max=0 101 real 0m1.111s user 0m0.009s sys 0m0.368s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff96825a65c800.statahead_max=2048 llite.lustre-ffff96825a65c800.statahead_max=2048 101 real 0m0.468s user 0m0.007s sys 0m0.293s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 4.23 seconds: 212.98 ops/second llite.lustre-ffff96825a65c800.statahead_max=0 1001 real 0m10.078s user 0m0.033s sys 0m3.153s 'ls -l' 1000 files without statahead: 10 sec llite.lustre-ffff96825a65c800.statahead_max=2048 llite.lustre-ffff96825a65c800.statahead_max=2048 1001 real 0m3.653s user 0m0.015s sys 0m1.489s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 8 seconds llite.lustre-ffff96825a65c800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327671.510995518 secs.nsecs start_time: 1789327492.743048692 secs.nsecs elapsed_time: 178.767946826 secs.nsecs subreqs per batch batches %% cum %% 1: 1 16 16 2: 0 0 16 4: 1 16 33 8: 2 33 66 16: 0 0 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327671.511059329 secs.nsecs start_time: 1789327492.743139217 secs.nsecs elapsed_time: 178.767920112 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 1 12 37 16: 1 12 50 32: 0 0 50 64: 1 12 62 128: 1 12 75 256: 2 25 100 - open/close 2112 (time 1789327682.73 total 10.00 last 211.19) - open/close 4223 (time 1789327692.73 total 20.00 last 211.08) - open/close 6330 (time 1789327702.73 total 30.00 last 210.68) - open/close 8475 (time 1789327712.74 total 40.01 last 214.43) total: 10000 open/close in 47.26 seconds: 211.58 ops/second llite.lustre-ffff96825a65c800.statahead_batch_max=0 llite.lustre-ffff96825a65c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m24.817s user 0m0.103s sys 0m21.910s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff96825a65c800.statahead_batch_max=256 llite.lustre-ffff96825a65c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff96825a65c800.stats=clear mdc.lustre-MDT0001-mdc-ffff96825a65c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m32.042s user 0m0.129s sys 0m22.587s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327785.612311831 secs.nsecs start_time: 1789327748.040504066 secs.nsecs elapsed_time: 37.571807765 secs.nsecs subreqs per batch batches %% cum %% 1: 86 50 50 2: 14 8 58 4: 5 2 61 8: 2 1 62 16: 1 0 63 32: 25 14 77 64: 2 1 78 128: 0 0 78 256: 36 21 100 mdc.lustre-MDT0001-mdc-ffff96825a65c800.batch_stats= snapshot_time: 1789327785.612366023 secs.nsecs start_time: 1789327748.040611928 secs.nsecs elapsed_time: 37.571754095 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff96825a65c800.statahead_batch_max=64 llite.lustre-ffff96825a65c800.statahead_max=128