== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:20:41 (1787566841) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=0 llite.lustre-ffff9efe0a084000.statahead_max=32 llite.lustre-ffff9efe0a084000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=0 total: 100 open/close in 2.91 seconds: 34.36 ops/second llite.lustre-ffff9efe0a084000.statahead_max=0 101 real 0m2.721s user 0m0.007s sys 0m0.317s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9efe0a084000.statahead_max=32 llite.lustre-ffff9efe0a084000.statahead_max=32 101 real 0m1.475s user 0m0.005s sys 0m0.200s 'ls -l' 100 files with statahead: 2 sec statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 30 fname_total: 111 hit_total: 3992 miss_total: 1739 swrong=2 ewrong=2 - open/close 567 (time 1787566871.69 total 10.01 last 56.65) total: 900 open/close in 15.42 seconds: 58.37 ops/second llite.lustre-ffff9efe0a084000.statahead_max=0 1001 real 0m24.523s user 0m0.034s sys 0m2.879s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9efe0a084000.statahead_max=32 llite.lustre-ffff9efe0a084000.statahead_max=32 1001 real 0m13.574s user 0m0.034s sys 0m1.438s 'ls -l' 1000 files with statahead: 13 sec statahead total: 142 statahead wrong: 2 agl total: 136 list_total: 31 fname_total: 111 hit_total: 4989 miss_total: 1742 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff9efe0a084000.statahead_stats= statahead total: 142 statahead wrong: 2 agl total: 136 list_total: 31 fname_total: 111 hit_total: 4989 miss_total: 1742 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats= snapshot_time: 1787566960.921633233 secs.nsecs start_time: 1787566847.463981389 secs.nsecs elapsed_time: 113.457651844 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 637 (time 1787566974.03 total 10.02 last 63.57) - open/close 1171 (time 1787566984.03 total 20.02 last 53.40) - open/close 1827 (time 1787566994.04 total 30.03 last 65.52) - open/close 2287 (time 1787567004.07 total 40.06 last 45.88) - open/close 2844 (time 1787567014.09 total 50.08 last 55.59) - open/close 3381 (time 1787567024.09 total 60.08 last 53.69) - open/close 4120 (time 1787567034.09 total 70.08 last 73.88) - open/close 4721 (time 1787567044.11 total 80.10 last 60.01) - open/close 5404 (time 1787567054.11 total 90.11 last 68.25) - open/close 6117 (time 1787567064.13 total 100.12 last 71.21) - open/close 6781 (time 1787567074.13 total 110.12 last 66.37) - open/close 7457 (time 1787567084.14 total 120.13 last 67.57) - open/close 8120 (time 1787567094.14 total 130.13 last 66.28) - open/close 8748 (time 1787567104.14 total 140.13 last 62.80) - open/close 9331 (time 1787567114.15 total 150.14 last 58.24) - open/close 9917 (time 1787567124.15 total 160.14 last 58.59) total: 10000 open/close in 161.31 seconds: 61.99 ops/second llite.lustre-ffff9efe0a084000.statahead_batch_max=0 llite.lustre-ffff9efe0a084000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m21.199s user 0m0.317s sys 0m14.654s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9efe0a084000.statahead_batch_max=32 llite.lustre-ffff9efe0a084000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m4.114s user 0m0.320s sys 0m14.052s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats= snapshot_time: 1787567355.170029820 secs.nsecs start_time: 1787567218.108194681 secs.nsecs elapsed_time: 137.061835139 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=0 llite.lustre-ffff9efe0a084000.statahead_max=2048 llite.lustre-ffff9efe0a084000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.46 ops/second llite.lustre-ffff9efe0a084000.statahead_max=0 101 real 0m2.658s user 0m0.004s sys 0m0.347s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9efe0a084000.statahead_max=2048 llite.lustre-ffff9efe0a084000.statahead_max=2048 101 real 0m1.079s user 0m0.007s sys 0m0.195s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 568 (time 1787567711.49 total 10.01 last 56.76) total: 900 open/close in 16.78 seconds: 53.62 ops/second llite.lustre-ffff9efe0a084000.statahead_max=0 1001 real 0m23.845s user 0m0.036s sys 0m3.268s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9efe0a084000.statahead_max=2048 llite.lustre-ffff9efe0a084000.statahead_max=2048 1001 real 0m9.629s user 0m0.037s sys 0m1.515s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9efe0a084000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats= snapshot_time: 1787567793.542775111 secs.nsecs start_time: 1787567357.933191434 secs.nsecs elapsed_time: 435.609583677 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 655 (time 1787567806.69 total 10.01 last 65.45) - open/close 1223 (time 1787567816.70 total 20.02 last 56.74) - open/close 1824 (time 1787567826.70 total 30.02 last 60.08) - open/close 2388 (time 1787567836.71 total 40.02 last 56.38) - open/close 3046 (time 1787567846.72 total 50.04 last 65.67) - open/close 3642 (time 1787567856.73 total 60.04 last 59.59) - open/close 4305 (time 1787567866.73 total 70.05 last 66.27) - open/close 4856 (time 1787567876.73 total 80.05 last 55.09) - open/close 5471 (time 1787567886.74 total 90.06 last 61.45) - open/close 6132 (time 1787567896.74 total 100.06 last 66.08) - open/close 6662 (time 1787567906.75 total 110.07 last 52.98) - open/close 7281 (time 1787567916.78 total 120.10 last 61.67) - open/close 7907 (time 1787567926.79 total 130.11 last 62.57) - open/close 8447 (time 1787567936.80 total 140.12 last 53.95) - open/close 9035 (time 1787567946.80 total 150.12 last 58.79) - open/close 9653 (time 1787567956.81 total 160.13 last 61.73) total: 10000 open/close in 167.12 seconds: 59.84 ops/second llite.lustre-ffff9efe0a084000.statahead_batch_max=0 llite.lustre-ffff9efe0a084000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m16.210s user 0m0.336s sys 0m15.159s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9efe0a084000.statahead_batch_max=256 llite.lustre-ffff9efe0a084000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9efe0a084000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.265s user 0m0.298s sys 0m14.129s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9efe0a084000.batch_stats= snapshot_time: 1787568119.354230220 secs.nsecs start_time: 1787568050.164254182 secs.nsecs elapsed_time: 69.189976038 secs.nsecs subreqs per batch batches %% cum %% 1: 76 36 36 2: 30 14 50 4: 25 11 62 8: 12 5 68 16: 9 4 72 32: 19 9 81 64: 2 0 82 128: 2 0 83 256: 35 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9efe0a084000.statahead_batch_max=64 llite.lustre-ffff9efe0a084000.statahead_max=128