== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:39:17 (1788442757) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=0 llite.lustre-ffff94ce828bc000.statahead_max=32 llite.lustre-ffff94ce828bc000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=0 total: 100 open/close in 2.93 seconds: 34.10 ops/second llite.lustre-ffff94ce828bc000.statahead_max=0 101 real 0m2.474s user 0m0.004s sys 0m0.308s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff94ce828bc000.statahead_max=32 llite.lustre-ffff94ce828bc000.statahead_max=32 101 real 0m1.107s user 0m0.004s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 128 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 68 hit_total: 4099 miss_total: 1334 swrong=2 ewrong=2 - open/close 632 (time 1788442786.15 total 10.00 last 63.20) total: 900 open/close in 14.33 seconds: 62.82 ops/second llite.lustre-ffff94ce828bc000.statahead_max=0 1001 real 0m24.232s user 0m0.044s sys 0m3.109s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff94ce828bc000.statahead_max=32 llite.lustre-ffff94ce828bc000.statahead_max=32 1001 real 0m10.636s user 0m0.038s sys 0m1.365s 'ls -l' 1000 files with statahead: 10 sec statahead total: 129 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 68 hit_total: 5095 miss_total: 1338 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff94ce828bc000.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 68 hit_total: 5095 miss_total: 1338 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats= snapshot_time: 1788442861.584056164 secs.nsecs start_time: 1788442763.080482942 secs.nsecs elapsed_time: 98.503573222 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 540 (time 1788442874.29 total 10.00 last 53.98) - open/close 1048 (time 1788442884.31 total 20.03 last 50.69) - open/close 1655 (time 1788442894.32 total 30.03 last 60.66) - open/close 2247 (time 1788442904.33 total 40.04 last 59.15) - open/close 2763 (time 1788442914.34 total 50.05 last 51.54) - open/close 3197 (time 1788442924.35 total 60.06 last 43.35) - open/close 3724 (time 1788442934.35 total 70.06 last 52.70) - open/close 4296 (time 1788442944.36 total 80.07 last 57.15) - open/close 4922 (time 1788442954.37 total 90.08 last 62.53) - open/close 5521 (time 1788442964.38 total 100.09 last 59.86) - open/close 6132 (time 1788442974.39 total 110.10 last 61.05) - open/close 6720 (time 1788442984.40 total 120.11 last 58.75) - open/close 7299 (time 1788442994.43 total 130.14 last 57.72) - open/close 7853 (time 1788443004.45 total 140.17 last 55.25) - open/close 8429 (time 1788443014.49 total 150.20 last 57.42) - open/close 9062 (time 1788443024.50 total 160.21 last 63.22) - open/close 9664 (time 1788443034.52 total 170.23 last 60.09) total: 10000 open/close in 176.23 seconds: 56.74 ops/second llite.lustre-ffff94ce828bc000.statahead_batch_max=0 llite.lustre-ffff94ce828bc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.928s user 0m0.351s sys 0m15.570s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94ce828bc000.statahead_batch_max=32 llite.lustre-ffff94ce828bc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m51.608s user 0m0.344s sys 0m14.164s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats= snapshot_time: 1788443240.722566753 secs.nsecs start_time: 1788443118.811839351 secs.nsecs elapsed_time: 121.910727402 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=0 llite.lustre-ffff94ce828bc000.statahead_max=2048 llite.lustre-ffff94ce828bc000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=0 total: 100 open/close in 2.55 seconds: 39.20 ops/second llite.lustre-ffff94ce828bc000.statahead_max=0 101 real 0m2.509s user 0m0.008s sys 0m0.311s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff94ce828bc000.statahead_max=2048 llite.lustre-ffff94ce828bc000.statahead_max=2048 101 real 0m0.855s user 0m0.006s sys 0m0.132s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 416 (time 1788443588.71 total 10.00 last 41.60) total: 900 open/close in 18.25 seconds: 49.31 ops/second llite.lustre-ffff94ce828bc000.statahead_max=0 1001 real 0m26.154s user 0m0.023s sys 0m2.954s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff94ce828bc000.statahead_max=2048 llite.lustre-ffff94ce828bc000.statahead_max=2048 1001 real 0m7.906s user 0m0.029s sys 0m1.416s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff94ce828bc000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats= snapshot_time: 1788443669.789174634 secs.nsecs start_time: 1788443243.181741780 secs.nsecs elapsed_time: 426.607432854 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 480 (time 1788443683.51 total 10.00 last 47.98) - open/close 1033 (time 1788443693.52 total 20.01 last 55.25) - open/close 1575 (time 1788443703.52 total 30.02 last 54.16) - open/close 2029 (time 1788443713.53 total 40.03 last 45.35) - open/close 2541 (time 1788443723.54 total 50.04 last 51.15) - open/close 3137 (time 1788443733.55 total 60.04 last 59.58) - open/close 3768 (time 1788443743.55 total 70.05 last 63.06) - open/close 4350 (time 1788443753.55 total 80.05 last 58.19) - open/close 4968 (time 1788443763.58 total 90.08 last 61.64) - open/close 5454 (time 1788443773.59 total 100.09 last 48.56) - open/close 6002 (time 1788443783.59 total 110.09 last 54.80) - open/close 6665 (time 1788443793.60 total 120.09 last 66.27) - open/close 7229 (time 1788443803.61 total 130.11 last 56.30) - open/close 7777 (time 1788443813.62 total 140.11 last 54.79) - open/close 8360 (time 1788443823.63 total 150.12 last 58.23) - open/close 9043 (time 1788443833.63 total 160.12 last 68.30) - open/close 9651 (time 1788443843.64 total 170.14 last 60.73) total: 10000 open/close in 175.46 seconds: 56.99 ops/second llite.lustre-ffff94ce828bc000.statahead_batch_max=0 llite.lustre-ffff94ce828bc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.685s user 0m0.363s sys 0m15.507s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94ce828bc000.statahead_batch_max=256 llite.lustre-ffff94ce828bc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94ce828bc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.233s user 0m0.258s sys 0m13.535s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff94ce828bc000.batch_stats= snapshot_time: 1788443981.686382016 secs.nsecs start_time: 1788443921.507551266 secs.nsecs elapsed_time: 60.178830750 secs.nsecs subreqs per batch batches %% cum %% 1: 99 40 40 2: 29 11 52 4: 25 10 62 8: 16 6 69 16: 15 6 75 32: 18 7 83 64: 6 2 85 128: 1 0 86 256: 34 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff94ce828bc000.statahead_batch_max=64 llite.lustre-ffff94ce828bc000.statahead_max=128