== sanity test 123ad: Verify batching statahead works correctly ========================================================== 08:42:44 (1789303364) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=0 llite.lustre-ffff9fa1c5079000.statahead_max=32 llite.lustre-ffff9fa1c5079000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=0 total: 100 open/close in 2.80 seconds: 35.78 ops/second llite.lustre-ffff9fa1c5079000.statahead_max=0 101 real 0m2.249s user 0m0.004s sys 0m0.324s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9fa1c5079000.statahead_max=32 llite.lustre-ffff9fa1c5079000.statahead_max=32 101 real 0m1.063s user 0m0.005s sys 0m0.159s 'ls -l' 100 files with statahead: 1 sec statahead total: 102 statahead wrong: 2 agl total: 96 list_total: 29 fname_total: 72 hit_total: 3612 miss_total: 1512 swrong=2 ewrong=2 - open/close 687 (time 1789303392.75 total 10.02 last 68.55) total: 900 open/close in 13.14 seconds: 68.51 ops/second llite.lustre-ffff9fa1c5079000.statahead_max=0 1001 real 0m22.890s user 0m0.025s sys 0m2.926s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9fa1c5079000.statahead_max=32 llite.lustre-ffff9fa1c5079000.statahead_max=32 1001 real 0m10.189s user 0m0.032s sys 0m1.410s 'ls -l' 1000 files with statahead: 10 sec statahead total: 103 statahead wrong: 2 agl total: 97 list_total: 30 fname_total: 72 hit_total: 4609 miss_total: 1515 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9fa1c5079000.statahead_stats= statahead total: 103 statahead wrong: 2 agl total: 97 list_total: 30 fname_total: 72 hit_total: 4609 miss_total: 1515 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats= snapshot_time: 1789303463.845623819 secs.nsecs start_time: 1789303369.730608698 secs.nsecs elapsed_time: 94.115015121 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 697 (time 1789303476.33 total 10.01 last 69.66) - open/close 1350 (time 1789303486.34 total 20.02 last 65.24) - open/close 2023 (time 1789303496.36 total 30.03 last 67.19) - open/close 2668 (time 1789303506.37 total 40.05 last 64.41) - open/close 3377 (time 1789303516.38 total 50.06 last 70.80) - open/close 4134 (time 1789303526.42 total 60.10 last 75.40) - open/close 4821 (time 1789303536.44 total 70.11 last 68.61) - open/close 5560 (time 1789303546.44 total 80.12 last 73.87) - open/close 6254 (time 1789303556.45 total 90.12 last 69.36) - open/close 6951 (time 1789303566.45 total 100.13 last 69.66) - open/close 7707 (time 1789303576.47 total 110.15 last 75.45) - open/close 8386 (time 1789303586.47 total 120.15 last 67.88) - open/close 9051 (time 1789303596.48 total 130.15 last 66.48) - open/close 9730 (time 1789303606.48 total 140.15 last 67.89) total: 10000 open/close in 143.48 seconds: 69.70 ops/second llite.lustre-ffff9fa1c5079000.statahead_batch_max=0 llite.lustre-ffff9fa1c5079000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.401s user 0m0.317s sys 0m14.662s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9fa1c5079000.statahead_batch_max=32 llite.lustre-ffff9fa1c5079000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.777s user 0m0.303s sys 0m13.986s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats= snapshot_time: 1789303799.223931952 secs.nsecs start_time: 1789303686.313128394 secs.nsecs elapsed_time: 112.910803558 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=0 llite.lustre-ffff9fa1c5079000.statahead_max=2048 llite.lustre-ffff9fa1c5079000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=0 total: 100 open/close in 2.88 seconds: 34.69 ops/second llite.lustre-ffff9fa1c5079000.statahead_max=0 101 real 0m2.027s user 0m0.004s sys 0m0.295s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9fa1c5079000.statahead_max=2048 llite.lustre-ffff9fa1c5079000.statahead_max=2048 101 real 0m1.117s user 0m0.002s sys 0m0.193s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 714 (time 1789304128.90 total 10.01 last 71.32) total: 900 open/close in 13.16 seconds: 68.37 ops/second llite.lustre-ffff9fa1c5079000.statahead_max=0 1001 real 0m22.298s user 0m0.025s sys 0m2.921s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9fa1c5079000.statahead_max=2048 llite.lustre-ffff9fa1c5079000.statahead_max=2048 1001 real 0m6.774s user 0m0.036s sys 0m1.609s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9fa1c5079000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats= snapshot_time: 1789304197.641277018 secs.nsecs start_time: 1789303802.212539725 secs.nsecs elapsed_time: 395.428737293 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 688 (time 1789304210.72 total 10.01 last 68.74) - open/close 1405 (time 1789304220.73 total 20.02 last 71.64) - open/close 2091 (time 1789304230.74 total 30.02 last 68.55) - open/close 2888 (time 1789304240.74 total 40.03 last 79.69) - open/close 3580 (time 1789304250.74 total 50.03 last 69.18) - open/close 4331 (time 1789304260.76 total 60.05 last 74.95) - open/close 5089 (time 1789304270.76 total 70.05 last 75.77) - open/close 5832 (time 1789304280.77 total 80.06 last 74.25) - open/close 6552 (time 1789304290.78 total 90.07 last 71.93) - open/close 7280 (time 1789304300.80 total 100.09 last 72.65) - open/close 7997 (time 1789304310.82 total 110.11 last 71.56) - open/close 8703 (time 1789304320.82 total 120.11 last 70.60) - open/close 9472 (time 1789304330.83 total 130.12 last 76.86) total: 10000 open/close in 137.00 seconds: 72.99 ops/second llite.lustre-ffff9fa1c5079000.statahead_batch_max=0 llite.lustre-ffff9fa1c5079000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.274s user 0m0.315s sys 0m14.771s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9fa1c5079000.statahead_batch_max=256 llite.lustre-ffff9fa1c5079000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.569s user 0m0.292s sys 0m14.917s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9fa1c5079000.batch_stats= snapshot_time: 1789304480.454629176 secs.nsecs start_time: 1789304411.669917756 secs.nsecs elapsed_time: 68.784711420 secs.nsecs subreqs per batch batches %% cum %% 1: 140 49 49 2: 32 11 61 4: 26 9 70 8: 17 6 76 16: 11 3 80 32: 15 5 85 64: 3 1 86 128: 3 1 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9fa1c5079000.statahead_batch_max=64 llite.lustre-ffff9fa1c5079000.statahead_max=128