== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:08:17 (1788469697) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=0 llite.lustre-ffff88b604da6000.statahead_max=32 llite.lustre-ffff88b604da6000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=0 total: 100 open/close in 2.78 seconds: 36.01 ops/second llite.lustre-ffff88b604da6000.statahead_max=0 101 real 0m2.190s user 0m0.002s sys 0m0.308s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff88b604da6000.statahead_max=32 llite.lustre-ffff88b604da6000.statahead_max=32 101 real 0m0.900s user 0m0.003s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 125 statahead wrong: 2 agl total: 119 list_total: 30 fname_total: 76 hit_total: 3889 miss_total: 1186 swrong=2 ewrong=2 - open/close 719 (time 1788469725.39 total 10.00 last 71.86) total: 900 open/close in 12.86 seconds: 69.97 ops/second llite.lustre-ffff88b604da6000.statahead_max=0 1001 real 0m21.534s user 0m0.034s sys 0m2.766s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff88b604da6000.statahead_max=32 llite.lustre-ffff88b604da6000.statahead_max=32 1001 real 0m9.605s user 0m0.027s sys 0m1.370s 'ls -l' 1000 files with statahead: 9 sec statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 76 hit_total: 4886 miss_total: 1189 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff88b604da6000.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 76 hit_total: 4886 miss_total: 1189 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats= snapshot_time: 1788469799.395598897 secs.nsecs start_time: 1788469702.631527417 secs.nsecs elapsed_time: 96.764071480 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 661 (time 1788469812.49 total 10.02 last 65.98) - open/close 1278 (time 1788469822.50 total 20.03 last 61.63) - open/close 1895 (time 1788469832.50 total 30.03 last 61.69) - open/close 2577 (time 1788469842.51 total 40.04 last 68.14) - open/close 3228 (time 1788469852.53 total 50.06 last 65.00) - open/close 3959 (time 1788469862.54 total 60.07 last 72.98) - open/close 4693 (time 1788469872.55 total 70.08 last 73.33) - open/close 5432 (time 1788469882.56 total 80.09 last 73.82) - open/close 6079 (time 1788469892.58 total 90.11 last 64.57) - open/close 6662 (time 1788469902.60 total 100.13 last 58.22) - open/close 7138 (time 1788469912.61 total 110.13 last 47.57) - open/close 7856 (time 1788469922.61 total 120.14 last 71.78) - open/close 8527 (time 1788469932.61 total 130.14 last 67.07) - open/close 9241 (time 1788469942.63 total 140.16 last 71.26) - open/close 9811 (time 1788469952.64 total 150.17 last 56.94) total: 10000 open/close in 153.36 seconds: 65.21 ops/second llite.lustre-ffff88b604da6000.statahead_batch_max=0 llite.lustre-ffff88b604da6000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m26.967s user 0m0.300s sys 0m14.047s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88b604da6000.statahead_batch_max=32 llite.lustre-ffff88b604da6000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.401s user 0m0.294s sys 0m13.278s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats= snapshot_time: 1788470172.986404536 secs.nsecs start_time: 1788470054.031891143 secs.nsecs elapsed_time: 118.954513393 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=0 llite.lustre-ffff88b604da6000.statahead_max=2048 llite.lustre-ffff88b604da6000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=0 total: 100 open/close in 2.88 seconds: 34.73 ops/second llite.lustre-ffff88b604da6000.statahead_max=0 101 real 0m2.169s user 0m0.006s sys 0m0.290s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff88b604da6000.statahead_max=2048 llite.lustre-ffff88b604da6000.statahead_max=2048 101 real 0m0.786s user 0m0.006s sys 0m0.146s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 722 (time 1788470523.50 total 10.00 last 72.20) total: 900 open/close in 12.44 seconds: 72.34 ops/second llite.lustre-ffff88b604da6000.statahead_max=0 1001 real 0m22.726s user 0m0.037s sys 0m2.909s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff88b604da6000.statahead_max=2048 llite.lustre-ffff88b604da6000.statahead_max=2048 1001 real 0m8.518s user 0m0.029s sys 0m1.353s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff88b604da6000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats= snapshot_time: 1788470600.767458497 secs.nsecs start_time: 1788470175.621022063 secs.nsecs elapsed_time: 425.146436434 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 675 (time 1788470613.49 total 10.00 last 67.48) - open/close 1336 (time 1788470623.49 total 20.01 last 66.08) - open/close 2060 (time 1788470633.50 total 30.01 last 72.39) - open/close 2774 (time 1788470643.50 total 40.01 last 71.36) - open/close 3449 (time 1788470653.51 total 50.02 last 67.46) - open/close 4174 (time 1788470663.51 total 60.03 last 72.44) - open/close 4907 (time 1788470673.54 total 70.05 last 73.12) - open/close 5585 (time 1788470683.55 total 80.06 last 67.74) - open/close 6337 (time 1788470693.57 total 90.08 last 75.08) - open/close 7046 (time 1788470703.57 total 100.08 last 70.89) - open/close 7740 (time 1788470713.57 total 110.08 last 69.38) - open/close 8456 (time 1788470723.58 total 120.09 last 71.55) - open/close 9151 (time 1788470733.59 total 130.10 last 69.43) - open/close 9780 (time 1788470743.59 total 140.10 last 62.87) total: 10000 open/close in 144.09 seconds: 69.40 ops/second llite.lustre-ffff88b604da6000.statahead_batch_max=0 llite.lustre-ffff88b604da6000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.761s user 0m0.314s sys 0m14.210s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88b604da6000.statahead_batch_max=256 llite.lustre-ffff88b604da6000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88b604da6000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.009s user 0m0.305s sys 0m13.765s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88b604da6000.batch_stats= snapshot_time: 1788470891.109645535 secs.nsecs start_time: 1788470823.368165027 secs.nsecs elapsed_time: 67.741480508 secs.nsecs subreqs per batch batches %% cum %% 1: 113 48 48 2: 33 14 62 4: 11 4 67 8: 8 3 70 16: 11 4 75 32: 18 7 82 64: 4 1 84 128: 1 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88b604da6000.statahead_batch_max=64 llite.lustre-ffff88b604da6000.statahead_max=128