== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:39:47 (1788845987) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=0 llite.lustre-ffff9282b3e06800.statahead_max=32 llite.lustre-ffff9282b3e06800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=0 total: 100 open/close in 2.46 seconds: 40.60 ops/second llite.lustre-ffff9282b3e06800.statahead_max=0 101 real 0m2.254s user 0m0.008s sys 0m0.312s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9282b3e06800.statahead_max=32 llite.lustre-ffff9282b3e06800.statahead_max=32 101 real 0m1.191s user 0m0.001s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 29 fname_total: 69 hit_total: 4105 miss_total: 1343 swrong=2 ewrong=2 - open/close 710 (time 1788846014.92 total 10.02 last 70.87) total: 900 open/close in 12.71 seconds: 70.84 ops/second llite.lustre-ffff9282b3e06800.statahead_max=0 1001 real 0m21.755s user 0m0.037s sys 0m2.836s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9282b3e06800.statahead_max=32 llite.lustre-ffff9282b3e06800.statahead_max=32 1001 real 0m11.292s user 0m0.029s sys 0m1.379s 'ls -l' 1000 files with statahead: 11 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 69 hit_total: 5101 miss_total: 1347 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9282b3e06800.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 30 fname_total: 69 hit_total: 5101 miss_total: 1347 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats= snapshot_time: 1788846088.127193523 secs.nsecs start_time: 1788845991.978693430 secs.nsecs elapsed_time: 96.148500093 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 679 (time 1788846100.54 total 10.01 last 67.84) - open/close 1448 (time 1788846110.55 total 20.01 last 76.87) - open/close 2187 (time 1788846120.56 total 30.03 last 73.80) - open/close 2953 (time 1788846130.56 total 40.03 last 76.60) - open/close 3712 (time 1788846140.57 total 50.04 last 75.83) - open/close 4444 (time 1788846150.57 total 60.04 last 73.18) - open/close 5167 (time 1788846160.59 total 70.05 last 72.20) - open/close 5898 (time 1788846170.60 total 80.07 last 72.99) - open/close 6633 (time 1788846180.63 total 90.09 last 73.32) - open/close 7348 (time 1788846190.63 total 100.09 last 71.48) - open/close 8071 (time 1788846200.64 total 110.11 last 72.20) - open/close 8822 (time 1788846210.65 total 120.11 last 75.07) - open/close 9418 (time 1788846220.65 total 130.11 last 59.58) total: 10000 open/close in 139.74 seconds: 71.56 ops/second llite.lustre-ffff9282b3e06800.statahead_batch_max=0 llite.lustre-ffff9282b3e06800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.122s user 0m0.328s sys 0m14.135s unbatched statahead RPC=10005 hit count=9998 llite.lustre-ffff9282b3e06800.statahead_batch_max=32 llite.lustre-ffff9282b3e06800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m46.229s user 0m0.298s sys 0m13.280s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats= snapshot_time: 1788846423.408405992 secs.nsecs start_time: 1788846305.840316679 secs.nsecs elapsed_time: 117.568089313 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=0 llite.lustre-ffff9282b3e06800.statahead_max=2048 llite.lustre-ffff9282b3e06800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=0 total: 100 open/close in 3.37 seconds: 29.64 ops/second llite.lustre-ffff9282b3e06800.statahead_max=0 101 real 0m2.551s user 0m0.009s sys 0m0.310s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9282b3e06800.statahead_max=2048 llite.lustre-ffff9282b3e06800.statahead_max=2048 101 real 0m1.128s user 0m0.006s sys 0m0.188s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 652 (time 1788846765.23 total 10.01 last 65.16) total: 900 open/close in 14.03 seconds: 64.16 ops/second llite.lustre-ffff9282b3e06800.statahead_max=0 1001 real 0m22.499s user 0m0.038s sys 0m3.040s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff9282b3e06800.statahead_max=2048 llite.lustre-ffff9282b3e06800.statahead_max=2048 1001 real 0m6.076s user 0m0.036s sys 0m1.339s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9282b3e06800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats= snapshot_time: 1788846832.838601715 secs.nsecs start_time: 1788846425.980649403 secs.nsecs elapsed_time: 406.857952312 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 636 (time 1788846845.36 total 10.00 last 63.59) - open/close 1301 (time 1788846855.37 total 20.01 last 66.44) - open/close 1919 (time 1788846865.39 total 30.02 last 61.72) - open/close 2598 (time 1788846875.39 total 40.03 last 67.88) - open/close 3295 (time 1788846885.40 total 50.04 last 69.63) - open/close 3963 (time 1788846895.41 total 60.04 last 66.75) - open/close 4588 (time 1788846905.42 total 70.06 last 62.41) - open/close 5218 (time 1788846915.44 total 80.07 last 62.92) - open/close 5851 (time 1788846925.44 total 90.07 last 63.29) - open/close 6509 (time 1788846935.46 total 100.10 last 65.64) - open/close 7087 (time 1788846945.46 total 110.10 last 57.79) - open/close 7755 (time 1788846955.46 total 120.10 last 66.80) - open/close 8420 (time 1788846965.47 total 130.11 last 66.44) - open/close 9073 (time 1788846975.49 total 140.12 last 65.21) - open/close 9606 (time 1788846985.50 total 150.13 last 53.26) total: 10000 open/close in 156.52 seconds: 63.89 ops/second llite.lustre-ffff9282b3e06800.statahead_batch_max=0 llite.lustre-ffff9282b3e06800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.146s user 0m0.326s sys 0m14.536s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9282b3e06800.statahead_batch_max=256 llite.lustre-ffff9282b3e06800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9282b3e06800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.031s user 0m0.314s sys 0m14.300s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9282b3e06800.batch_stats= snapshot_time: 1788847132.853279220 secs.nsecs start_time: 1788847063.453725456 secs.nsecs elapsed_time: 69.399553764 secs.nsecs subreqs per batch batches %% cum %% 1: 115 43 43 2: 42 15 58 4: 19 7 65 8: 17 6 72 16: 16 5 78 32: 16 5 84 64: 8 2 87 128: 0 0 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9282b3e06800.statahead_batch_max=64 llite.lustre-ffff9282b3e06800.statahead_max=128