== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:25:53 (1788834353) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=0 llite.lustre-ffff8df1e0e9a000.statahead_max=32 llite.lustre-ffff8df1e0e9a000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=0 total: 100 open/close in 2.31 seconds: 43.23 ops/second llite.lustre-ffff8df1e0e9a000.statahead_max=0 101 real 0m1.891s user 0m0.004s sys 0m0.277s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8df1e0e9a000.statahead_max=32 llite.lustre-ffff8df1e0e9a000.statahead_max=32 101 real 0m0.896s user 0m0.004s sys 0m0.138s 'ls -l' 100 files with statahead: 1 sec statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 29 fname_total: 83 hit_total: 3586 miss_total: 1368 swrong=2 ewrong=2 - open/close 600 (time 1788834379.95 total 10.00 last 59.98) total: 900 open/close in 15.04 seconds: 59.82 ops/second llite.lustre-ffff8df1e0e9a000.statahead_max=0 1001 real 0m21.462s user 0m0.031s sys 0m2.881s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8df1e0e9a000.statahead_max=32 llite.lustre-ffff8df1e0e9a000.statahead_max=32 1001 real 0m10.815s user 0m0.029s sys 0m1.300s 'ls -l' 1000 files with statahead: 11 sec statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 83 hit_total: 4584 miss_total: 1370 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff8df1e0e9a000.statahead_stats= statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 83 hit_total: 4584 miss_total: 1370 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats= snapshot_time: 1788834455.047252746 secs.nsecs start_time: 1788834357.993235360 secs.nsecs elapsed_time: 97.054017386 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 8 8 16: 0 0 8 32: 34 91 100 - open/close 872 (time 1788834467.71 total 10.01 last 87.13) - open/close 1656 (time 1788834477.71 total 20.01 last 78.37) - open/close 2166 (time 1788834487.71 total 30.01 last 50.98) - open/close 2828 (time 1788834497.72 total 40.02 last 66.15) - open/close 3460 (time 1788834507.73 total 50.03 last 63.16) - open/close 4261 (time 1788834517.74 total 60.04 last 80.01) - open/close 5151 (time 1788834527.74 total 70.04 last 88.96) - open/close 5980 (time 1788834537.74 total 80.05 last 82.88) - open/close 6856 (time 1788834547.75 total 90.05 last 87.53) - open/close 7490 (time 1788834557.76 total 100.06 last 63.37) - open/close 8226 (time 1788834567.76 total 110.06 last 73.58) - open/close 9035 (time 1788834577.76 total 120.07 last 80.86) - open/close 9956 (time 1788834587.77 total 130.07 last 92.07) total: 10000 open/close in 130.60 seconds: 76.57 ops/second llite.lustre-ffff8df1e0e9a000.statahead_batch_max=0 llite.lustre-ffff8df1e0e9a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.515s user 0m0.313s sys 0m13.443s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8df1e0e9a000.statahead_batch_max=32 llite.lustre-ffff8df1e0e9a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.554s user 0m0.254s sys 0m12.882s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats= snapshot_time: 1788834761.891350185 secs.nsecs start_time: 1788834652.225491451 secs.nsecs elapsed_time: 109.665858734 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=0 llite.lustre-ffff8df1e0e9a000.statahead_max=2048 llite.lustre-ffff8df1e0e9a000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=0 total: 100 open/close in 2.88 seconds: 34.73 ops/second llite.lustre-ffff8df1e0e9a000.statahead_max=0 101 real 0m1.869s user 0m0.006s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8df1e0e9a000.statahead_max=2048 llite.lustre-ffff8df1e0e9a000.statahead_max=2048 101 real 0m1.114s user 0m0.004s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 698 (time 1788835063.47 total 10.01 last 69.76) total: 900 open/close in 12.99 seconds: 69.29 ops/second llite.lustre-ffff8df1e0e9a000.statahead_max=0 1001 real 0m20.692s user 0m0.034s sys 0m2.818s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff8df1e0e9a000.statahead_max=2048 llite.lustre-ffff8df1e0e9a000.statahead_max=2048 1001 real 0m9.695s user 0m0.029s sys 0m1.247s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff8df1e0e9a000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats= snapshot_time: 1788835133.776151733 secs.nsecs start_time: 1788834764.486101534 secs.nsecs elapsed_time: 369.290050199 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 736 (time 1788835146.40 total 10.02 last 73.48) - open/close 1553 (time 1788835156.41 total 20.02 last 81.65) - open/close 2408 (time 1788835166.41 total 30.03 last 85.47) - open/close 3179 (time 1788835176.43 total 40.04 last 77.00) - open/close 3962 (time 1788835186.43 total 50.05 last 78.25) - open/close 4765 (time 1788835196.44 total 60.06 last 80.22) - open/close 5512 (time 1788835206.45 total 70.06 last 74.65) - open/close 6372 (time 1788835216.45 total 80.06 last 85.99) - open/close 7287 (time 1788835226.45 total 90.07 last 91.49) - open/close 8158 (time 1788835236.46 total 100.07 last 87.03) - open/close 8841 (time 1788835246.47 total 110.08 last 68.26) - open/close 9691 (time 1788835256.48 total 120.09 last 84.93) total: 10000 open/close in 123.42 seconds: 81.02 ops/second llite.lustre-ffff8df1e0e9a000.statahead_batch_max=0 llite.lustre-ffff8df1e0e9a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.650s user 0m0.289s sys 0m13.664s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8df1e0e9a000.statahead_batch_max=256 llite.lustre-ffff8df1e0e9a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.465s user 0m0.312s sys 0m13.213s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8df1e0e9a000.batch_stats= snapshot_time: 1788835394.240887585 secs.nsecs start_time: 1788835335.469584431 secs.nsecs elapsed_time: 58.771303154 secs.nsecs subreqs per batch batches %% cum %% 1: 164 46 46 2: 62 17 64 4: 41 11 76 8: 12 3 79 16: 15 4 84 32: 18 5 89 64: 3 0 90 128: 1 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8df1e0e9a000.statahead_batch_max=64 llite.lustre-ffff8df1e0e9a000.statahead_max=128