== sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:58:37 (1788469117) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=0 llite.lustre-ffff9a44c3312800.statahead_max=32 llite.lustre-ffff9a44c3312800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=0 total: 100 open/close in 2.46 seconds: 40.68 ops/second llite.lustre-ffff9a44c3312800.statahead_max=0 101 real 0m2.940s user 0m0.005s sys 0m0.349s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9a44c3312800.statahead_max=32 llite.lustre-ffff9a44c3312800.statahead_max=32 101 real 0m1.138s user 0m0.007s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 107 statahead wrong: 2 agl total: 100 list_total: 32 fname_total: 75 hit_total: 3968 miss_total: 1644 swrong=2 ewrong=2 - open/close 572 (time 1788469147.90 total 10.03 last 57.00) total: 900 open/close in 15.59 seconds: 57.75 ops/second llite.lustre-ffff9a44c3312800.statahead_max=0 1001 real 0m21.120s user 0m0.030s sys 0m2.902s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9a44c3312800.statahead_max=32 llite.lustre-ffff9a44c3312800.statahead_max=32 1001 real 0m8.702s user 0m0.021s sys 0m1.372s 'ls -l' 1000 files with statahead: 8 sec statahead total: 108 statahead wrong: 2 agl total: 101 list_total: 33 fname_total: 75 hit_total: 4965 miss_total: 1647 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9a44c3312800.statahead_stats= statahead total: 108 statahead wrong: 2 agl total: 101 list_total: 33 fname_total: 75 hit_total: 4965 miss_total: 1647 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats= snapshot_time: 1788469215.501899346 secs.nsecs start_time: 1788469123.386745249 secs.nsecs elapsed_time: 92.115154097 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 694 (time 1788469227.77 total 10.03 last 69.18) - open/close 1393 (time 1788469237.79 total 20.05 last 69.80) - open/close 2065 (time 1788469247.79 total 30.05 last 67.17) - open/close 2730 (time 1788469257.80 total 40.05 last 66.48) - open/close 3463 (time 1788469267.80 total 50.06 last 73.27) - open/close 4109 (time 1788469277.80 total 60.06 last 64.59) - open/close 4771 (time 1788469287.82 total 70.08 last 66.06) - open/close 5368 (time 1788469297.82 total 80.08 last 59.69) - open/close 5953 (time 1788469307.83 total 90.08 last 58.48) - open/close 6665 (time 1788469317.84 total 100.09 last 71.14) - open/close 7363 (time 1788469327.85 total 110.11 last 69.68) - open/close 8023 (time 1788469337.86 total 120.12 last 65.95) - open/close 8601 (time 1788469347.88 total 130.14 last 57.69) - open/close 9214 (time 1788469357.88 total 140.14 last 61.29) - open/close 9779 (time 1788469367.88 total 150.14 last 56.48) total: 10000 open/close in 153.70 seconds: 65.06 ops/second llite.lustre-ffff9a44c3312800.statahead_batch_max=0 llite.lustre-ffff9a44c3312800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.022s user 0m0.339s sys 0m14.829s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a44c3312800.statahead_batch_max=32 llite.lustre-ffff9a44c3312800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.021s user 0m0.297s sys 0m13.588s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats= snapshot_time: 1788469556.433460488 secs.nsecs start_time: 1788469438.729107197 secs.nsecs elapsed_time: 117.704353291 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=0 llite.lustre-ffff9a44c3312800.statahead_max=2048 llite.lustre-ffff9a44c3312800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=0 total: 100 open/close in 3.39 seconds: 29.46 ops/second llite.lustre-ffff9a44c3312800.statahead_max=0 101 real 0m2.461s user 0m0.004s sys 0m0.311s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9a44c3312800.statahead_max=2048 llite.lustre-ffff9a44c3312800.statahead_max=2048 101 real 0m0.659s user 0m0.004s sys 0m0.147s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 538 (time 1788469919.08 total 10.02 last 53.70) total: 900 open/close in 16.52 seconds: 54.48 ops/second llite.lustre-ffff9a44c3312800.statahead_max=0 1001 real 0m25.255s user 0m0.030s sys 0m2.968s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9a44c3312800.statahead_max=2048 llite.lustre-ffff9a44c3312800.statahead_max=2048 1001 real 0m7.311s user 0m0.026s sys 0m1.479s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9a44c3312800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats= snapshot_time: 1788470003.922437741 secs.nsecs start_time: 1788469559.039193011 secs.nsecs elapsed_time: 444.883244730 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 513 (time 1788470016.76 total 10.01 last 51.25) - open/close 1013 (time 1788470026.79 total 20.04 last 49.86) - open/close 1408 (time 1788470036.81 total 30.05 last 39.43) - open/close 2020 (time 1788470046.83 total 40.08 last 61.05) - open/close 2571 (time 1788470056.85 total 50.10 last 55.01) - open/close 3145 (time 1788470066.86 total 60.11 last 57.34) - open/close 3809 (time 1788470076.87 total 70.12 last 66.32) - open/close 4505 (time 1788470086.87 total 80.12 last 69.59) - open/close 5153 (time 1788470096.88 total 90.13 last 64.74) - open/close 5774 (time 1788470106.89 total 100.13 last 62.07) - open/close 6316 (time 1788470116.89 total 110.14 last 54.17) - open/close 6872 (time 1788470126.90 total 120.14 last 55.59) - open/close 7319 (time 1788470136.91 total 130.16 last 44.63) - open/close 7747 (time 1788470146.92 total 140.16 last 42.78) - open/close 8193 (time 1788470156.92 total 150.17 last 44.57) - open/close 8646 (time 1788470166.94 total 160.18 last 45.24) - open/close 9139 (time 1788470176.96 total 170.20 last 49.20) - open/close 9601 (time 1788470186.96 total 180.21 last 46.16) total: 10000 open/close in 189.31 seconds: 52.82 ops/second llite.lustre-ffff9a44c3312800.statahead_batch_max=0 llite.lustre-ffff9a44c3312800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m24.722s user 0m0.322s sys 0m14.955s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a44c3312800.statahead_batch_max=256 llite.lustre-ffff9a44c3312800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a44c3312800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.428s user 0m0.283s sys 0m14.089s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9a44c3312800.batch_stats= snapshot_time: 1788470367.091177646 secs.nsecs start_time: 1788470298.016259517 secs.nsecs elapsed_time: 69.074918129 secs.nsecs subreqs per batch batches %% cum %% 1: 84 40 40 2: 25 11 51 4: 16 7 59 8: 16 7 67 16: 13 6 73 32: 15 7 80 64: 4 1 82 128: 2 0 83 256: 35 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9a44c3312800.statahead_batch_max=64 llite.lustre-ffff9a44c3312800.statahead_max=128