== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:26:04 (1788848764) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=0 llite.lustre-ffff97e182f3e000.statahead_max=32 llite.lustre-ffff97e182f3e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=0 total: 100 open/close in 2.31 seconds: 43.38 ops/second llite.lustre-ffff97e182f3e000.statahead_max=0 101 real 0m1.882s user 0m0.004s sys 0m0.298s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97e182f3e000.statahead_max=32 llite.lustre-ffff97e182f3e000.statahead_max=32 101 real 0m0.948s user 0m0.002s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 128 statahead wrong: 2 agl total: 120 list_total: 30 fname_total: 97 hit_total: 4515 miss_total: 773 swrong=2 ewrong=2 - open/close 705 (time 1788848790.02 total 10.02 last 70.34) total: 900 open/close in 12.40 seconds: 72.58 ops/second llite.lustre-ffff97e182f3e000.statahead_max=0 1001 real 0m20.800s user 0m0.029s sys 0m2.896s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff97e182f3e000.statahead_max=32 llite.lustre-ffff97e182f3e000.statahead_max=32 1001 real 0m11.112s user 0m0.033s sys 0m1.382s 'ls -l' 1000 files with statahead: 11 sec statahead total: 129 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 97 hit_total: 5509 miss_total: 779 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff97e182f3e000.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 97 hit_total: 5509 miss_total: 779 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats= snapshot_time: 1788848859.301114151 secs.nsecs start_time: 1788848768.453239308 secs.nsecs elapsed_time: 90.847874843 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 5 5 16: 3 7 12 32: 34 87 100 - open/close 648 (time 1788848871.52 total 10.00 last 64.77) - open/close 1430 (time 1788848881.53 total 20.01 last 78.12) - open/close 2211 (time 1788848891.53 total 30.02 last 78.06) - open/close 3006 (time 1788848901.54 total 40.02 last 79.48) - open/close 3794 (time 1788848911.57 total 50.05 last 78.58) - open/close 4605 (time 1788848921.58 total 60.06 last 81.01) - open/close 5352 (time 1788848931.59 total 70.07 last 74.64) - open/close 6231 (time 1788848941.60 total 80.09 last 87.75) - open/close 7021 (time 1788848951.61 total 90.09 last 78.96) - open/close 7826 (time 1788848961.62 total 100.11 last 80.37) - open/close 8578 (time 1788848971.63 total 110.11 last 75.19) - open/close 9459 (time 1788848981.63 total 120.12 last 88.06) total: 10000 open/close in 126.69 seconds: 78.93 ops/second llite.lustre-ffff97e182f3e000.statahead_batch_max=0 llite.lustre-ffff97e182f3e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.165s user 0m0.327s sys 0m14.241s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97e182f3e000.statahead_batch_max=32 llite.lustre-ffff97e182f3e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m28.939s user 0m0.317s sys 0m13.386s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats= snapshot_time: 1788849145.968465654 secs.nsecs start_time: 1788849048.263641567 secs.nsecs elapsed_time: 97.704824087 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=0 llite.lustre-ffff97e182f3e000.statahead_max=2048 llite.lustre-ffff97e182f3e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=0 total: 100 open/close in 2.61 seconds: 38.34 ops/second llite.lustre-ffff97e182f3e000.statahead_max=0 101 real 0m2.124s user 0m0.004s sys 0m0.306s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97e182f3e000.statahead_max=2048 llite.lustre-ffff97e182f3e000.statahead_max=2048 101 real 0m0.803s user 0m0.003s sys 0m0.169s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 756 (time 1788849435.67 total 10.00 last 75.59) total: 900 open/close in 11.98 seconds: 75.16 ops/second llite.lustre-ffff97e182f3e000.statahead_max=0 1001 real 0m19.589s user 0m0.035s sys 0m2.893s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff97e182f3e000.statahead_max=2048 llite.lustre-ffff97e182f3e000.statahead_max=2048 1001 real 0m6.481s user 0m0.035s sys 0m1.593s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff97e182f3e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats= snapshot_time: 1788849493.107979833 secs.nsecs start_time: 1788849149.017757957 secs.nsecs elapsed_time: 344.090221876 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 808 (time 1788849505.21 total 10.01 last 80.74) - open/close 1604 (time 1788849515.23 total 20.02 last 79.50) - open/close 2402 (time 1788849525.23 total 30.02 last 79.77) - open/close 3255 (time 1788849535.24 total 40.03 last 85.21) - open/close 4086 (time 1788849545.26 total 50.05 last 82.96) - open/close 4940 (time 1788849555.27 total 60.06 last 85.29) - open/close 5749 (time 1788849565.28 total 70.07 last 80.85) - open/close 6548 (time 1788849575.28 total 80.07 last 79.88) - open/close 7369 (time 1788849585.29 total 90.08 last 82.03) - open/close 8177 (time 1788849595.29 total 100.08 last 80.78) - open/close 8990 (time 1788849605.30 total 110.09 last 81.23) - open/close 9792 (time 1788849615.31 total 120.10 last 80.10) total: 10000 open/close in 122.51 seconds: 81.63 ops/second llite.lustre-ffff97e182f3e000.statahead_batch_max=0 llite.lustre-ffff97e182f3e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.904s user 0m0.337s sys 0m19.766s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97e182f3e000.statahead_batch_max=256 llite.lustre-ffff97e182f3e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97e182f3e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m38.435s user 0m0.325s sys 0m16.563s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff97e182f3e000.batch_stats= snapshot_time: 1788849709.952230583 secs.nsecs start_time: 1788849663.986191262 secs.nsecs elapsed_time: 45.966039321 secs.nsecs subreqs per batch batches %% cum %% 1: 151 52 52 2: 27 9 61 4: 28 9 71 8: 19 6 77 16: 10 3 81 32: 15 5 86 64: 4 1 87 128: 0 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff97e182f3e000.statahead_batch_max=64 llite.lustre-ffff97e182f3e000.statahead_max=128