== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:52:04 (1788933124) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=0 llite.lustre-ffff9ed98315b000.statahead_max=32 llite.lustre-ffff9ed98315b000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.79 ops/second llite.lustre-ffff9ed98315b000.statahead_max=0 101 real 0m2.055s user 0m0.004s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9ed98315b000.statahead_max=32 llite.lustre-ffff9ed98315b000.statahead_max=32 101 real 0m1.181s user 0m0.007s sys 0m0.164s 'ls -l' 100 files with statahead: 1 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 29 fname_total: 64 hit_total: 3799 miss_total: 1227 swrong=2 ewrong=2 - open/close 737 (time 1788933152.63 total 10.01 last 73.66) total: 900 open/close in 12.28 seconds: 73.31 ops/second llite.lustre-ffff9ed98315b000.statahead_max=0 1001 real 0m24.035s user 0m0.036s sys 0m2.852s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9ed98315b000.statahead_max=32 llite.lustre-ffff9ed98315b000.statahead_max=32 1001 real 0m8.330s user 0m0.033s sys 0m1.295s 'ls -l' 1000 files with statahead: 9 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 30 fname_total: 64 hit_total: 4796 miss_total: 1230 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9ed98315b000.statahead_stats= statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 30 fname_total: 64 hit_total: 4796 miss_total: 1230 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats= snapshot_time: 1788933219.238551215 secs.nsecs start_time: 1788933129.321785343 secs.nsecs elapsed_time: 89.916765872 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 786 (time 1788933231.52 total 10.01 last 78.51) - open/close 1564 (time 1788933241.54 total 20.03 last 77.65) - open/close 2297 (time 1788933251.55 total 30.04 last 73.24) - open/close 3034 (time 1788933261.56 total 40.05 last 73.61) - open/close 3756 (time 1788933271.56 total 50.05 last 72.19) - open/close 4391 (time 1788933281.56 total 60.05 last 63.49) - open/close 5001 (time 1788933291.56 total 70.06 last 60.98) - open/close 5694 (time 1788933301.57 total 80.06 last 69.26) - open/close 6419 (time 1788933311.58 total 90.07 last 72.43) - open/close 7151 (time 1788933321.60 total 100.09 last 73.06) - open/close 7917 (time 1788933331.60 total 110.10 last 76.56) - open/close 8703 (time 1788933341.61 total 120.11 last 78.52) - open/close 9452 (time 1788933351.62 total 130.12 last 74.84) total: 10000 open/close in 136.21 seconds: 73.42 ops/second llite.lustre-ffff9ed98315b000.statahead_batch_max=0 llite.lustre-ffff9ed98315b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.206s user 0m0.306s sys 0m14.258s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ed98315b000.statahead_batch_max=32 llite.lustre-ffff9ed98315b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m51.754s user 0m0.293s sys 0m12.962s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats= snapshot_time: 1788933550.799569828 secs.nsecs start_time: 1788933426.916050544 secs.nsecs elapsed_time: 123.883519284 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=0 llite.lustre-ffff9ed98315b000.statahead_max=2048 llite.lustre-ffff9ed98315b000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=0 total: 100 open/close in 2.38 seconds: 42.07 ops/second llite.lustre-ffff9ed98315b000.statahead_max=0 101 real 0m2.464s user 0m0.005s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9ed98315b000.statahead_max=2048 llite.lustre-ffff9ed98315b000.statahead_max=2048 101 real 0m0.760s user 0m0.002s sys 0m0.170s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 662 (time 1788933838.77 total 10.00 last 66.17) total: 900 open/close in 13.14 seconds: 68.51 ops/second llite.lustre-ffff9ed98315b000.statahead_max=0 1001 real 0m19.957s user 0m0.031s sys 0m2.815s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9ed98315b000.statahead_max=2048 llite.lustre-ffff9ed98315b000.statahead_max=2048 1001 real 0m6.118s user 0m0.033s sys 0m1.429s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9ed98315b000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats= snapshot_time: 1788933900.055525654 secs.nsecs start_time: 1788933553.710938371 secs.nsecs elapsed_time: 346.344587283 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 1 10 30 32: 2 20 50 64: 1 10 60 128: 0 0 60 256: 4 40 100 - open/close 785 (time 1788933912.43 total 10.01 last 78.43) - open/close 1547 (time 1788933922.45 total 20.03 last 76.06) - open/close 2320 (time 1788933932.46 total 30.04 last 77.23) - open/close 3142 (time 1788933942.46 total 40.04 last 82.20) - open/close 4007 (time 1788933952.47 total 50.04 last 86.42) - open/close 4808 (time 1788933962.48 total 60.05 last 80.04) - open/close 5437 (time 1788933972.49 total 70.06 last 62.85) - open/close 6181 (time 1788933982.49 total 80.06 last 74.39) - open/close 7007 (time 1788933992.49 total 90.07 last 82.53) - open/close 7702 (time 1788934002.50 total 100.07 last 69.50) - open/close 8516 (time 1788934012.50 total 110.07 last 81.40) - open/close 9268 (time 1788934022.51 total 120.09 last 75.07) total: 10000 open/close in 129.71 seconds: 77.09 ops/second llite.lustre-ffff9ed98315b000.statahead_batch_max=0 llite.lustre-ffff9ed98315b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.174s user 0m0.306s sys 0m14.401s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ed98315b000.statahead_batch_max=256 llite.lustre-ffff9ed98315b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ed98315b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.002s user 0m0.318s sys 0m14.483s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ed98315b000.batch_stats= snapshot_time: 1788934161.936527649 secs.nsecs start_time: 1788934099.899237343 secs.nsecs elapsed_time: 62.037290306 secs.nsecs subreqs per batch batches %% cum %% 1: 93 42 42 2: 26 11 53 4: 18 8 61 8: 12 5 67 16: 16 7 74 32: 18 8 82 64: 3 1 84 128: 0 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ed98315b000.statahead_batch_max=64 llite.lustre-ffff9ed98315b000.statahead_max=128