== sanity test 123ad: Verify batching statahead works correctly ========================================================== 08:12:57 (1787400777) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=0 llite.lustre-ffff9f12d25a9000.statahead_max=32 llite.lustre-ffff9f12d25a9000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=0 total: 100 open/close in 3.16 seconds: 31.63 ops/second llite.lustre-ffff9f12d25a9000.statahead_max=0 101 real 0m2.727s user 0m0.005s sys 0m0.364s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f12d25a9000.statahead_max=32 llite.lustre-ffff9f12d25a9000.statahead_max=32 101 real 0m1.313s user 0m0.003s sys 0m0.165s 'ls -l' 100 files with statahead: 1 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 30 fname_total: 77 hit_total: 3672 miss_total: 1494 swrong=2 ewrong=2 - open/close 595 (time 1787400807.68 total 10.01 last 59.42) total: 900 open/close in 14.33 seconds: 62.82 ops/second llite.lustre-ffff9f12d25a9000.statahead_max=0 1001 real 0m26.039s user 0m0.036s sys 0m3.381s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9f12d25a9000.statahead_max=32 llite.lustre-ffff9f12d25a9000.statahead_max=32 1001 real 0m12.993s user 0m0.022s sys 0m1.595s 'ls -l' 1000 files with statahead: 13 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 31 fname_total: 77 hit_total: 4669 miss_total: 1497 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9f12d25a9000.statahead_stats= statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 31 fname_total: 77 hit_total: 4669 miss_total: 1497 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats= snapshot_time: 1787400888.052930545 secs.nsecs start_time: 1787400783.493299604 secs.nsecs elapsed_time: 104.559630941 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 644 (time 1787400900.31 total 10.01 last 64.33) - open/close 1314 (time 1787400910.34 total 20.03 last 66.85) - open/close 1910 (time 1787400920.34 total 30.03 last 59.59) - open/close 2573 (time 1787400930.36 total 40.05 last 66.17) - open/close 3199 (time 1787400940.37 total 50.07 last 62.51) - open/close 3895 (time 1787400950.37 total 60.07 last 69.58) - open/close 4535 (time 1787400960.39 total 70.09 last 63.91) - open/close 5213 (time 1787400970.40 total 80.10 last 67.70) - open/close 5831 (time 1787400980.40 total 90.10 last 61.80) - open/close 6475 (time 1787400990.42 total 100.11 last 64.30) - open/close 7110 (time 1787401000.42 total 110.12 last 63.50) - open/close 7744 (time 1787401010.42 total 120.12 last 63.38) - open/close 8345 (time 1787401020.44 total 130.14 last 59.96) - open/close 8984 (time 1787401030.46 total 140.16 last 63.81) - open/close 9564 (time 1787401040.47 total 150.17 last 57.92) total: 10000 open/close in 156.99 seconds: 63.70 ops/second llite.lustre-ffff9f12d25a9000.statahead_batch_max=0 llite.lustre-ffff9f12d25a9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.189s user 0m0.343s sys 0m17.085s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f12d25a9000.statahead_batch_max=32 llite.lustre-ffff9f12d25a9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m0.815s user 0m0.322s sys 0m15.738s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats= snapshot_time: 1787401266.074255058 secs.nsecs start_time: 1787401129.671930271 secs.nsecs elapsed_time: 136.402324787 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=0 llite.lustre-ffff9f12d25a9000.statahead_max=2048 llite.lustre-ffff9f12d25a9000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=0 total: 100 open/close in 3.63 seconds: 27.53 ops/second llite.lustre-ffff9f12d25a9000.statahead_max=0 101 real 0m2.525s user 0m0.006s sys 0m0.338s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9f12d25a9000.statahead_max=2048 llite.lustre-ffff9f12d25a9000.statahead_max=2048 101 real 0m0.970s user 0m0.004s sys 0m0.164s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 629 (time 1787401657.40 total 10.00 last 62.89) total: 900 open/close in 14.46 seconds: 62.22 ops/second llite.lustre-ffff9f12d25a9000.statahead_max=0 1001 real 0m27.957s user 0m0.032s sys 0m3.269s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff9f12d25a9000.statahead_max=2048 llite.lustre-ffff9f12d25a9000.statahead_max=2048 1001 real 0m8.399s user 0m0.038s sys 0m1.852s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9f12d25a9000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats= snapshot_time: 1787401743.235031884 secs.nsecs start_time: 1787401268.328787462 secs.nsecs elapsed_time: 474.906244422 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 588 (time 1787401756.83 total 10.00 last 58.79) - open/close 1193 (time 1787401766.84 total 20.02 last 60.42) - open/close 1817 (time 1787401776.85 total 30.02 last 62.38) - open/close 2468 (time 1787401786.85 total 40.02 last 65.08) - open/close 3068 (time 1787401796.85 total 50.03 last 59.98) - open/close 3723 (time 1787401806.85 total 60.03 last 65.49) - open/close 4288 (time 1787401816.89 total 70.06 last 56.31) - open/close 4886 (time 1787401826.91 total 80.08 last 59.68) - open/close 5506 (time 1787401836.91 total 90.09 last 61.97) - open/close 6098 (time 1787401846.91 total 100.09 last 59.19) - open/close 6586 (time 1787401856.92 total 110.09 last 48.77) - open/close 7089 (time 1787401866.93 total 120.10 last 50.25) - open/close 7575 (time 1787401876.93 total 130.10 last 48.60) - open/close 8143 (time 1787401886.94 total 140.11 last 56.75) - open/close 8739 (time 1787401896.95 total 150.12 last 59.55) - open/close 9319 (time 1787401906.96 total 160.13 last 57.95) - open/close 9916 (time 1787401916.97 total 170.14 last 59.62) total: 10000 open/close in 171.71 seconds: 58.24 ops/second llite.lustre-ffff9f12d25a9000.statahead_batch_max=0 llite.lustre-ffff9f12d25a9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.948s user 0m0.284s sys 0m16.921s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9f12d25a9000.statahead_batch_max=256 llite.lustre-ffff9f12d25a9000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.187s user 0m0.307s sys 0m17.334s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9f12d25a9000.batch_stats= snapshot_time: 1787402075.607065301 secs.nsecs start_time: 1787402004.713840432 secs.nsecs elapsed_time: 70.893224869 secs.nsecs subreqs per batch batches %% cum %% 1: 233 53 53 2: 58 13 66 4: 53 12 78 8: 23 5 84 16: 15 3 87 32: 18 4 91 64: 2 0 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9f12d25a9000.statahead_batch_max=64 llite.lustre-ffff9f12d25a9000.statahead_max=128