== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:01:38 (1789088498) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=0 llite.lustre-ffff9f4e80469000.statahead_max=32 llite.lustre-ffff9f4e80469000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=0 total: 100 open/close in 2.72 seconds: 36.73 ops/second llite.lustre-ffff9f4e80469000.statahead_max=0 101 real 0m2.877s user 0m0.009s sys 0m0.332s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f4e80469000.statahead_max=32 llite.lustre-ffff9f4e80469000.statahead_max=32 101 real 0m0.977s user 0m0.005s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 29 fname_total: 97 hit_total: 3997 miss_total: 1458 swrong=2 ewrong=2 - open/close 558 (time 1789088527.51 total 10.01 last 55.77) total: 900 open/close in 14.70 seconds: 61.23 ops/second llite.lustre-ffff9f4e80469000.statahead_max=0 1001 real 0m17.974s user 0m0.036s sys 0m2.876s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff9f4e80469000.statahead_max=32 llite.lustre-ffff9f4e80469000.statahead_max=32 1001 real 0m10.362s user 0m0.032s sys 0m1.582s 'ls -l' 1000 files with statahead: 10 sec statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 97 hit_total: 4995 miss_total: 1460 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9f4e80469000.statahead_stats= statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 97 hit_total: 4995 miss_total: 1460 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats= snapshot_time: 1789088595.113702471 secs.nsecs start_time: 1789088503.436352869 secs.nsecs elapsed_time: 91.677349602 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 767 (time 1789088607.62 total 10.01 last 76.64) - open/close 1676 (time 1789088617.62 total 20.01 last 90.87) - open/close 2554 (time 1789088627.63 total 30.01 last 87.78) - open/close 3411 (time 1789088637.63 total 40.02 last 85.65) - open/close 4305 (time 1789088647.64 total 50.02 last 89.36) - open/close 5131 (time 1789088657.64 total 60.03 last 82.56) - open/close 5980 (time 1789088667.65 total 70.04 last 84.82) - open/close 6890 (time 1789088677.65 total 80.04 last 90.98) - open/close 7738 (time 1789088687.69 total 90.08 last 84.46) - open/close 8526 (time 1789088697.69 total 100.08 last 78.80) - open/close 9341 (time 1789088707.71 total 110.10 last 81.40) total: 10000 open/close in 117.97 seconds: 84.77 ops/second llite.lustre-ffff9f4e80469000.statahead_batch_max=0 llite.lustre-ffff9f4e80469000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.024s user 0m0.292s sys 0m15.223s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f4e80469000.statahead_batch_max=32 llite.lustre-ffff9f4e80469000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.125s user 0m0.310s sys 0m14.618s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats= snapshot_time: 1789088913.351179272 secs.nsecs start_time: 1789088792.029901024 secs.nsecs elapsed_time: 121.321278248 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=0 llite.lustre-ffff9f4e80469000.statahead_max=2048 llite.lustre-ffff9f4e80469000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=0 total: 100 open/close in 2.44 seconds: 40.92 ops/second llite.lustre-ffff9f4e80469000.statahead_max=0 101 real 0m2.092s user 0m0.009s sys 0m0.320s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9f4e80469000.statahead_max=2048 llite.lustre-ffff9f4e80469000.statahead_max=2048 101 real 0m0.689s user 0m0.005s sys 0m0.166s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 758 (time 1789089217.55 total 10.00 last 75.77) total: 900 open/close in 11.79 seconds: 76.35 ops/second llite.lustre-ffff9f4e80469000.statahead_max=0 1001 real 0m19.721s user 0m0.034s sys 0m2.971s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9f4e80469000.statahead_max=2048 llite.lustre-ffff9f4e80469000.statahead_max=2048 1001 real 0m7.683s user 0m0.030s sys 0m1.547s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9f4e80469000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats= snapshot_time: 1789089283.927683765 secs.nsecs start_time: 1789088915.676191433 secs.nsecs elapsed_time: 368.251492332 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 857 (time 1789089296.35 total 10.01 last 85.62) - open/close 1726 (time 1789089306.36 total 20.01 last 86.86) - open/close 2516 (time 1789089316.36 total 30.02 last 78.96) - open/close 3267 (time 1789089326.37 total 40.02 last 75.07) - open/close 3984 (time 1789089336.37 total 50.03 last 71.69) - open/close 4809 (time 1789089346.37 total 60.03 last 82.48) - open/close 5661 (time 1789089356.38 total 70.04 last 85.11) - open/close 6457 (time 1789089366.39 total 80.05 last 79.50) - open/close 7274 (time 1789089376.40 total 90.06 last 81.64) - open/close 8139 (time 1789089386.40 total 100.06 last 86.47) - open/close 8877 (time 1789089396.41 total 110.06 last 73.78) - open/close 9778 (time 1789089406.41 total 120.07 last 90.08) total: 10000 open/close in 122.48 seconds: 81.65 ops/second llite.lustre-ffff9f4e80469000.statahead_batch_max=0 llite.lustre-ffff9f4e80469000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.881s user 0m0.326s sys 0m16.287s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f4e80469000.statahead_batch_max=256 llite.lustre-ffff9f4e80469000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f4e80469000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.172s user 0m0.304s sys 0m15.445s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f4e80469000.batch_stats= snapshot_time: 1789089548.644102230 secs.nsecs start_time: 1789089471.817166965 secs.nsecs elapsed_time: 76.826935265 secs.nsecs subreqs per batch batches %% cum %% 1: 253 48 48 2: 97 18 67 4: 63 12 79 8: 31 5 85 16: 24 4 90 32: 15 2 93 64: 2 0 93 128: 0 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f4e80469000.statahead_batch_max=64 llite.lustre-ffff9f4e80469000.statahead_max=128