== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:05:46 (1788642346) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=0 llite.lustre-ffff8d41067fc800.statahead_max=32 llite.lustre-ffff8d41067fc800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=0 total: 100 open/close in 3.19 seconds: 31.34 ops/second llite.lustre-ffff8d41067fc800.statahead_max=0 101 real 0m2.856s user 0m0.005s sys 0m0.342s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8d41067fc800.statahead_max=32 llite.lustre-ffff8d41067fc800.statahead_max=32 101 real 0m1.164s user 0m0.007s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 112 statahead wrong: 2 agl total: 107 list_total: 30 fname_total: 75 hit_total: 3731 miss_total: 1703 swrong=2 ewrong=2 - open/close 742 (time 1788642377.20 total 10.02 last 74.07) total: 900 open/close in 12.27 seconds: 73.38 ops/second llite.lustre-ffff8d41067fc800.statahead_max=0 1001 real 0m20.159s user 0m0.040s sys 0m3.175s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8d41067fc800.statahead_max=32 llite.lustre-ffff8d41067fc800.statahead_max=32 1001 real 0m9.484s user 0m0.027s sys 0m1.492s 'ls -l' 1000 files with statahead: 10 sec statahead total: 113 statahead wrong: 2 agl total: 108 list_total: 31 fname_total: 75 hit_total: 4729 miss_total: 1705 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8d41067fc800.statahead_stats= statahead total: 113 statahead wrong: 2 agl total: 108 list_total: 31 fname_total: 75 hit_total: 4729 miss_total: 1705 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats= snapshot_time: 1788642445.740319640 secs.nsecs start_time: 1788642352.608050392 secs.nsecs elapsed_time: 93.132269248 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 660 (time 1788642458.60 total 10.00 last 65.99) - open/close 1255 (time 1788642468.61 total 20.01 last 59.43) - open/close 1915 (time 1788642478.62 total 30.03 last 65.91) - open/close 2571 (time 1788642488.62 total 40.03 last 65.59) - open/close 3285 (time 1788642498.63 total 50.03 last 71.37) - open/close 3943 (time 1788642508.63 total 60.04 last 65.75) - open/close 4711 (time 1788642518.65 total 70.05 last 76.72) - open/close 5365 (time 1788642528.65 total 80.06 last 65.34) - open/close 6047 (time 1788642538.66 total 90.07 last 68.14) - open/close 6711 (time 1788642548.66 total 100.07 last 66.40) - open/close 7408 (time 1788642558.66 total 110.07 last 69.70) - open/close 8063 (time 1788642568.68 total 120.08 last 65.41) - open/close 8718 (time 1788642578.68 total 130.08 last 65.49) - open/close 9302 (time 1788642588.69 total 140.10 last 58.34) - open/close 9878 (time 1788642598.70 total 150.11 last 57.53) total: 10000 open/close in 152.06 seconds: 65.77 ops/second llite.lustre-ffff8d41067fc800.statahead_batch_max=0 llite.lustre-ffff8d41067fc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.284s user 0m0.360s sys 0m15.964s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d41067fc800.statahead_batch_max=32 llite.lustre-ffff8d41067fc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.527s user 0m0.359s sys 0m14.523s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats= snapshot_time: 1788642801.142811791 secs.nsecs start_time: 1788642679.019265198 secs.nsecs elapsed_time: 122.123546593 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=0 llite.lustre-ffff8d41067fc800.statahead_max=2048 llite.lustre-ffff8d41067fc800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=0 total: 100 open/close in 2.81 seconds: 35.60 ops/second llite.lustre-ffff8d41067fc800.statahead_max=0 101 real 0m2.317s user 0m0.005s sys 0m0.324s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8d41067fc800.statahead_max=2048 llite.lustre-ffff8d41067fc800.statahead_max=2048 101 real 0m0.640s user 0m0.005s sys 0m0.184s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 682 (time 1788643119.27 total 10.02 last 68.05) total: 900 open/close in 13.21 seconds: 68.12 ops/second llite.lustre-ffff8d41067fc800.statahead_max=0 1001 real 0m22.707s user 0m0.039s sys 0m3.008s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8d41067fc800.statahead_max=2048 llite.lustre-ffff8d41067fc800.statahead_max=2048 1001 real 0m6.948s user 0m0.031s sys 0m1.366s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8d41067fc800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats= snapshot_time: 1788643188.178550627 secs.nsecs start_time: 1788642804.041269616 secs.nsecs elapsed_time: 384.137281011 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 2 16 66 128: 0 0 66 256: 4 33 100 - open/close 645 (time 1788643201.84 total 10.01 last 64.45) - open/close 1236 (time 1788643211.84 total 20.01 last 59.09) - open/close 1904 (time 1788643221.84 total 30.01 last 66.79) - open/close 2620 (time 1788643231.85 total 40.02 last 71.54) - open/close 3356 (time 1788643241.86 total 50.02 last 73.57) - open/close 4072 (time 1788643251.87 total 60.04 last 71.50) - open/close 4844 (time 1788643261.87 total 70.04 last 77.20) - open/close 5641 (time 1788643271.87 total 80.04 last 79.65) - open/close 6382 (time 1788643281.88 total 90.05 last 74.04) - open/close 7078 (time 1788643291.90 total 100.07 last 69.50) - open/close 7698 (time 1788643301.90 total 110.07 last 61.97) - open/close 8391 (time 1788643311.92 total 120.08 last 69.21) - open/close 9026 (time 1788643321.93 total 130.10 last 63.43) - open/close 9656 (time 1788643331.94 total 140.11 last 62.93) total: 10000 open/close in 145.88 seconds: 68.55 ops/second llite.lustre-ffff8d41067fc800.statahead_batch_max=0 llite.lustre-ffff8d41067fc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.782s user 0m0.387s sys 0m16.216s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d41067fc800.statahead_batch_max=256 llite.lustre-ffff8d41067fc800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d41067fc800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.318s user 0m0.323s sys 0m15.178s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8d41067fc800.batch_stats= snapshot_time: 1788643471.680473043 secs.nsecs start_time: 1788643409.319014442 secs.nsecs elapsed_time: 62.361458601 secs.nsecs subreqs per batch batches %% cum %% 1: 89 43 43 2: 16 7 51 4: 20 9 61 8: 10 4 66 16: 6 2 69 32: 21 10 79 64: 6 2 82 128: 0 0 82 256: 35 17 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8d41067fc800.statahead_batch_max=64 llite.lustre-ffff8d41067fc800.statahead_max=128