== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:10:27 (1788750627) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=0 llite.lustre-ffff932ffc0b7800.statahead_max=32 llite.lustre-ffff932ffc0b7800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=0 total: 100 open/close in 2.54 seconds: 39.36 ops/second llite.lustre-ffff932ffc0b7800.statahead_max=0 101 real 0m2.234s user 0m0.007s sys 0m0.287s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff932ffc0b7800.statahead_max=32 llite.lustre-ffff932ffc0b7800.statahead_max=32 101 real 0m1.133s user 0m0.007s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 30 fname_total: 96 hit_total: 3557 miss_total: 1737 swrong=2 ewrong=2 - open/close 709 (time 1788750654.83 total 10.02 last 70.77) total: 900 open/close in 12.30 seconds: 73.20 ops/second llite.lustre-ffff932ffc0b7800.statahead_max=0 1001 real 0m24.046s user 0m0.031s sys 0m2.879s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff932ffc0b7800.statahead_max=32 llite.lustre-ffff932ffc0b7800.statahead_max=32 1001 real 0m11.883s user 0m0.028s sys 0m1.376s 'ls -l' 1000 files with statahead: 12 sec statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 96 hit_total: 4554 miss_total: 1740 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff932ffc0b7800.statahead_stats= statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 96 hit_total: 4554 miss_total: 1740 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats= snapshot_time: 1788750732.425397686 secs.nsecs start_time: 1788750632.497829437 secs.nsecs elapsed_time: 99.927568249 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 695 (time 1788750744.76 total 10.01 last 69.44) - open/close 1358 (time 1788750754.77 total 20.02 last 66.25) - open/close 2010 (time 1788750764.77 total 30.02 last 65.16) - open/close 2748 (time 1788750774.78 total 40.03 last 73.76) - open/close 3279 (time 1788750784.79 total 50.03 last 53.07) - open/close 3951 (time 1788750794.79 total 60.04 last 67.17) - open/close 4632 (time 1788750804.80 total 70.05 last 68.01) - open/close 5375 (time 1788750814.80 total 80.05 last 74.29) - open/close 6104 (time 1788750824.81 total 90.06 last 72.87) - open/close 6873 (time 1788750834.82 total 100.07 last 76.82) - open/close 7544 (time 1788750844.82 total 110.07 last 67.07) - open/close 8234 (time 1788750854.83 total 120.08 last 68.96) - open/close 8976 (time 1788750864.84 total 130.09 last 74.11) - open/close 9706 (time 1788750874.84 total 140.09 last 73.00) total: 10000 open/close in 144.07 seconds: 69.41 ops/second llite.lustre-ffff932ffc0b7800.statahead_batch_max=0 llite.lustre-ffff932ffc0b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.942s user 0m0.340s sys 0m15.156s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff932ffc0b7800.statahead_batch_max=32 llite.lustre-ffff932ffc0b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.244s user 0m0.288s sys 0m13.627s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats= snapshot_time: 1788751055.290653678 secs.nsecs start_time: 1788750942.479252397 secs.nsecs elapsed_time: 112.811401281 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=0 llite.lustre-ffff932ffc0b7800.statahead_max=2048 llite.lustre-ffff932ffc0b7800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=0 total: 100 open/close in 2.58 seconds: 38.72 ops/second llite.lustre-ffff932ffc0b7800.statahead_max=0 101 real 0m2.089s user 0m0.006s sys 0m0.303s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff932ffc0b7800.statahead_max=2048 llite.lustre-ffff932ffc0b7800.statahead_max=2048 101 real 0m0.595s user 0m0.008s sys 0m0.192s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 - open/close 775 (time 1788751359.81 total 10.00 last 77.47) total: 900 open/close in 11.74 seconds: 76.66 ops/second llite.lustre-ffff932ffc0b7800.statahead_max=0 1001 real 0m19.724s user 0m0.035s sys 0m2.850s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff932ffc0b7800.statahead_max=2048 llite.lustre-ffff932ffc0b7800.statahead_max=2048 1001 real 0m5.835s user 0m0.028s sys 0m1.438s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff932ffc0b7800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats= snapshot_time: 1788751413.373855840 secs.nsecs start_time: 1788751058.105867133 secs.nsecs elapsed_time: 355.267988707 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 746 (time 1788751425.77 total 10.01 last 74.55) - open/close 1486 (time 1788751435.77 total 20.01 last 73.95) - open/close 2253 (time 1788751445.78 total 30.02 last 76.63) - open/close 2991 (time 1788751455.79 total 40.03 last 73.78) - open/close 3744 (time 1788751465.79 total 50.03 last 75.23) - open/close 4514 (time 1788751475.81 total 60.05 last 76.87) - open/close 5264 (time 1788751485.82 total 70.06 last 74.92) - open/close 6026 (time 1788751495.84 total 80.08 last 76.07) - open/close 6827 (time 1788751505.85 total 90.09 last 80.02) - open/close 7600 (time 1788751515.86 total 100.10 last 77.22) - open/close 8427 (time 1788751525.86 total 110.10 last 82.67) - open/close 9243 (time 1788751535.88 total 120.12 last 81.50) total: 10000 open/close in 128.93 seconds: 77.56 ops/second llite.lustre-ffff932ffc0b7800.statahead_batch_max=0 llite.lustre-ffff932ffc0b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.859s user 0m0.325s sys 0m17.544s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff932ffc0b7800.statahead_batch_max=256 llite.lustre-ffff932ffc0b7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m40.230s user 0m0.317s sys 0m16.589s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff932ffc0b7800.batch_stats= snapshot_time: 1788751644.297420046 secs.nsecs start_time: 1788751595.005271382 secs.nsecs elapsed_time: 49.292148664 secs.nsecs subreqs per batch batches %% cum %% 1: 227 57 57 2: 38 9 66 4: 40 10 76 8: 19 4 81 16: 19 4 86 32: 17 4 90 64: 3 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff932ffc0b7800.statahead_batch_max=64 llite.lustre-ffff932ffc0b7800.statahead_max=128