== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:43:36 (1783975416) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=0 llite.lustre-ffff9acec52e5000.statahead_max=32 llite.lustre-ffff9acec52e5000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=0 total: 100 open/close in 3.26 seconds: 30.65 ops/second llite.lustre-ffff9acec52e5000.statahead_max=0 101 real 0m3.274s user 0m0.005s sys 0m0.360s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9acec52e5000.statahead_max=32 llite.lustre-ffff9acec52e5000.statahead_max=32 101 real 0m1.317s user 0m0.008s sys 0m0.162s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 72 hit_total: 4289 miss_total: 1281 swrong=2 ewrong=2 - open/close 691 (time 1783975448.39 total 10.01 last 69.05) total: 900 open/close in 12.33 seconds: 72.98 ops/second llite.lustre-ffff9acec52e5000.statahead_max=0 1001 real 0m21.280s user 0m0.027s sys 0m3.002s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9acec52e5000.statahead_max=32 llite.lustre-ffff9acec52e5000.statahead_max=32 1001 real 0m10.903s user 0m0.027s sys 0m1.392s 'ls -l' 1000 files with statahead: 11 sec statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 31 fname_total: 72 hit_total: 5286 miss_total: 1284 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff9acec52e5000.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 31 fname_total: 72 hit_total: 5286 miss_total: 1284 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats= snapshot_time: 1783975523.802796594 secs.nsecs start_time: 1783975422.553156245 secs.nsecs elapsed_time: 101.249640349 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 679 (time 1783975536.87 total 10.00 last 67.89) - open/close 1410 (time 1783975546.88 total 20.00 last 73.07) - open/close 2149 (time 1783975556.88 total 30.01 last 73.89) - open/close 2882 (time 1783975566.88 total 40.01 last 73.27) - open/close 3616 (time 1783975576.89 total 50.02 last 73.33) - open/close 4334 (time 1783975586.90 total 60.03 last 71.75) - open/close 5037 (time 1783975596.91 total 70.04 last 70.23) - open/close 5671 (time 1783975606.92 total 80.05 last 63.33) - open/close 6425 (time 1783975616.93 total 90.06 last 75.31) - open/close 6952 (time 1783975626.95 total 100.08 last 52.59) - open/close 7552 (time 1783975636.96 total 110.09 last 59.95) - open/close 8162 (time 1783975646.96 total 120.09 last 60.99) - open/close 8871 (time 1783975656.97 total 130.10 last 70.83) - open/close 9544 (time 1783975666.98 total 140.11 last 67.25) total: 10000 open/close in 147.15 seconds: 67.96 ops/second llite.lustre-ffff9acec52e5000.statahead_batch_max=0 llite.lustre-ffff9acec52e5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.426s user 0m0.336s sys 0m15.721s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9acec52e5000.statahead_batch_max=32 llite.lustre-ffff9acec52e5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m53.783s user 0m0.295s sys 0m14.814s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats= snapshot_time: 1783975893.195273946 secs.nsecs start_time: 1783975761.703259578 secs.nsecs elapsed_time: 131.492014368 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=0 llite.lustre-ffff9acec52e5000.statahead_max=2048 llite.lustre-ffff9acec52e5000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=0 total: 100 open/close in 1.92 seconds: 52.14 ops/second llite.lustre-ffff9acec52e5000.statahead_max=0 101 real 0m1.652s user 0m0.008s sys 0m0.318s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9acec52e5000.statahead_max=2048 llite.lustre-ffff9acec52e5000.statahead_max=2048 101 real 0m1.042s user 0m0.004s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 689 (time 1783976233.88 total 10.00 last 68.87) total: 900 open/close in 13.68 seconds: 65.78 ops/second llite.lustre-ffff9acec52e5000.statahead_max=0 1001 real 0m22.022s user 0m0.027s sys 0m3.104s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9acec52e5000.statahead_max=2048 llite.lustre-ffff9acec52e5000.statahead_max=2048 1001 real 0m8.302s user 0m0.030s sys 0m1.614s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9acec52e5000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats= snapshot_time: 1783976306.250120848 secs.nsecs start_time: 1783975896.098120892 secs.nsecs elapsed_time: 410.151999956 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 636 (time 1783976319.14 total 10.02 last 63.49) - open/close 1247 (time 1783976329.14 total 20.02 last 61.10) - open/close 1933 (time 1783976339.14 total 30.02 last 68.59) - open/close 2654 (time 1783976349.14 total 40.02 last 72.08) - open/close 3332 (time 1783976359.15 total 50.03 last 67.77) - open/close 4007 (time 1783976369.15 total 60.03 last 67.47) - open/close 4692 (time 1783976379.16 total 70.04 last 68.44) - open/close 5392 (time 1783976389.17 total 80.05 last 69.93) - open/close 6176 (time 1783976399.18 total 90.06 last 78.31) - open/close 6841 (time 1783976409.18 total 100.07 last 66.48) - open/close 7524 (time 1783976419.19 total 110.08 last 68.23) - open/close 8166 (time 1783976429.22 total 120.10 last 64.03) - open/close 8858 (time 1783976439.23 total 130.11 last 69.16) - open/close 9602 (time 1783976449.23 total 140.11 last 74.36) total: 10000 open/close in 145.23 seconds: 68.85 ops/second llite.lustre-ffff9acec52e5000.statahead_batch_max=0 llite.lustre-ffff9acec52e5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.147s user 0m0.297s sys 0m15.468s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9acec52e5000.statahead_batch_max=256 llite.lustre-ffff9acec52e5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9acec52e5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.558s user 0m0.281s sys 0m15.113s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9acec52e5000.batch_stats= snapshot_time: 1783976611.292840222 secs.nsecs start_time: 1783976538.251503113 secs.nsecs elapsed_time: 73.041337109 secs.nsecs subreqs per batch batches %% cum %% 1: 210 52 52 2: 64 15 68 4: 36 8 77 8: 20 4 82 16: 20 4 87 32: 15 3 90 64: 2 0 91 128: 1 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9acec52e5000.statahead_batch_max=64 llite.lustre-ffff9acec52e5000.statahead_max=128