== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:54:54 (1784051694) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=0 llite.lustre-ffff941761109000.statahead_max=32 llite.lustre-ffff941761109000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=0 total: 100 open/close in 3.08 seconds: 32.48 ops/second llite.lustre-ffff941761109000.statahead_max=0 101 real 0m2.431s user 0m0.004s sys 0m0.376s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff941761109000.statahead_max=32 llite.lustre-ffff941761109000.statahead_max=32 101 real 0m1.710s user 0m0.005s sys 0m0.212s 'ls -l' 100 files with statahead: 1 sec statahead total: 148 statahead wrong: 2 agl total: 141 list_total: 30 fname_total: 116 hit_total: 3570 miss_total: 1374 swrong=2 ewrong=2 - open/close 599 (time 1784051723.64 total 10.01 last 59.85) total: 900 open/close in 15.63 seconds: 57.57 ops/second llite.lustre-ffff941761109000.statahead_max=0 1001 real 0m26.640s user 0m0.030s sys 0m3.464s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff941761109000.statahead_max=32 llite.lustre-ffff941761109000.statahead_max=32 1001 real 0m12.632s user 0m0.044s sys 0m1.811s 'ls -l' 1000 files with statahead: 13 sec statahead total: 149 statahead wrong: 2 agl total: 142 list_total: 31 fname_total: 116 hit_total: 4567 miss_total: 1377 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff941761109000.statahead_stats= statahead total: 149 statahead wrong: 2 agl total: 142 list_total: 31 fname_total: 116 hit_total: 4567 miss_total: 1377 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats= snapshot_time: 1784051808.516968211 secs.nsecs start_time: 1784051698.902187371 secs.nsecs elapsed_time: 109.614780840 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 593 (time 1784051821.16 total 10.00 last 59.28) - open/close 1180 (time 1784051831.16 total 20.01 last 58.68) - open/close 1774 (time 1784051841.17 total 30.02 last 59.30) - open/close 2383 (time 1784051851.19 total 40.04 last 60.79) - open/close 3006 (time 1784051861.23 total 50.07 last 62.10) - open/close 3605 (time 1784051871.23 total 60.08 last 59.86) - open/close 4215 (time 1784051881.24 total 70.08 last 60.98) - open/close 4766 (time 1784051891.25 total 80.10 last 55.02) - open/close 5308 (time 1784051901.26 total 90.11 last 54.15) - open/close 5950 (time 1784051911.27 total 100.12 last 64.15) - open/close 6497 (time 1784051921.29 total 110.14 last 54.59) - open/close 7095 (time 1784051931.29 total 120.14 last 59.78) - open/close 7683 (time 1784051941.32 total 130.17 last 58.65) - open/close 8281 (time 1784051951.33 total 140.18 last 59.73) - open/close 8872 (time 1784051961.37 total 150.22 last 58.88) - open/close 9446 (time 1784051971.38 total 160.23 last 57.33) total: 10000 open/close in 169.70 seconds: 58.93 ops/second llite.lustre-ffff941761109000.statahead_batch_max=0 llite.lustre-ffff941761109000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.274s user 0m0.313s sys 0m16.613s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff941761109000.statahead_batch_max=32 llite.lustre-ffff941761109000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.397s user 0m0.332s sys 0m16.473s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats= snapshot_time: 1784052189.267579197 secs.nsecs start_time: 1784052063.552690876 secs.nsecs elapsed_time: 125.714888321 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=0 llite.lustre-ffff941761109000.statahead_max=2048 llite.lustre-ffff941761109000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=0 total: 100 open/close in 3.56 seconds: 28.08 ops/second llite.lustre-ffff941761109000.statahead_max=0 101 real 0m2.794s user 0m0.004s sys 0m0.415s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff941761109000.statahead_max=2048 llite.lustre-ffff941761109000.statahead_max=2048 101 real 0m0.990s user 0m0.004s sys 0m0.187s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 600 (time 1784052579.24 total 10.02 last 59.88) total: 900 open/close in 14.98 seconds: 60.09 ops/second llite.lustre-ffff941761109000.statahead_max=0 1001 real 0m28.740s user 0m0.046s sys 0m3.574s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffff941761109000.statahead_max=2048 llite.lustre-ffff941761109000.statahead_max=2048 1001 real 0m7.532s user 0m0.037s sys 0m1.892s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff941761109000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats= snapshot_time: 1784052665.816144581 secs.nsecs start_time: 1784052192.016113117 secs.nsecs elapsed_time: 473.800031464 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 1 10 60 128: 0 0 60 256: 4 40 100 - open/close 506 (time 1784052678.80 total 10.01 last 50.57) - open/close 1044 (time 1784052688.82 total 20.02 last 53.71) - open/close 1533 (time 1784052698.83 total 30.04 last 48.81) - open/close 2097 (time 1784052708.87 total 40.08 last 56.21) - open/close 2702 (time 1784052718.87 total 50.08 last 60.48) - open/close 3263 (time 1784052728.87 total 60.08 last 56.09) - open/close 3828 (time 1784052738.89 total 70.10 last 56.40) - open/close 4383 (time 1784052748.89 total 80.10 last 55.49) - open/close 4914 (time 1784052758.90 total 90.11 last 53.06) - open/close 5421 (time 1784052768.92 total 100.13 last 50.60) - open/close 5939 (time 1784052778.93 total 110.14 last 51.74) - open/close 6442 (time 1784052788.93 total 120.14 last 50.29) - open/close 6994 (time 1784052798.93 total 130.14 last 55.20) - open/close 7561 (time 1784052808.94 total 140.14 last 56.68) - open/close 8116 (time 1784052818.94 total 150.15 last 55.49) - open/close 8702 (time 1784052828.94 total 160.15 last 58.58) - open/close 9244 (time 1784052838.95 total 170.15 last 54.19) - open/close 9722 (time 1784052848.96 total 180.16 last 47.74) total: 10000 open/close in 185.56 seconds: 53.89 ops/second llite.lustre-ffff941761109000.statahead_batch_max=0 llite.lustre-ffff941761109000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m20.002s user 0m0.333s sys 0m17.079s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff941761109000.statahead_batch_max=256 llite.lustre-ffff941761109000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff941761109000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.936s user 0m0.301s sys 0m16.836s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff941761109000.batch_stats= snapshot_time: 1784053023.162134778 secs.nsecs start_time: 1784052948.053866222 secs.nsecs elapsed_time: 75.108268556 secs.nsecs subreqs per batch batches %% cum %% 1: 293 57 57 2: 71 14 71 4: 51 10 81 8: 24 4 86 16: 14 2 89 32: 14 2 92 64: 7 1 93 128: 0 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff941761109000.statahead_batch_max=64 llite.lustre-ffff941761109000.statahead_max=128