== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:52:43 (1788835963) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=0 llite.lustre-ffff9cc9461f4000.statahead_max=32 llite.lustre-ffff9cc9461f4000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=0 total: 100 open/close in 2.37 seconds: 42.20 ops/second llite.lustre-ffff9cc9461f4000.statahead_max=0 101 real 0m2.595s user 0m0.005s sys 0m0.286s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9cc9461f4000.statahead_max=32 llite.lustre-ffff9cc9461f4000.statahead_max=32 101 real 0m1.122s user 0m0.002s sys 0m0.142s 'ls -l' 100 files with statahead: 1 sec statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 30 fname_total: 83 hit_total: 3996 miss_total: 1528 swrong=2 ewrong=2 - open/close 768 (time 1788835991.50 total 10.00 last 76.78) total: 900 open/close in 11.60 seconds: 77.60 ops/second llite.lustre-ffff9cc9461f4000.statahead_max=0 1001 real 0m18.057s user 0m0.033s sys 0m2.832s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff9cc9461f4000.statahead_max=32 llite.lustre-ffff9cc9461f4000.statahead_max=32 1001 real 0m10.745s user 0m0.030s sys 0m1.367s 'ls -l' 1000 files with statahead: 11 sec statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 31 fname_total: 83 hit_total: 4995 miss_total: 1529 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9cc9461f4000.statahead_stats= statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 31 fname_total: 83 hit_total: 4995 miss_total: 1529 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats= snapshot_time: 1788836059.080918472 secs.nsecs start_time: 1788835968.613367228 secs.nsecs elapsed_time: 90.467551244 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 686 (time 1788836071.92 total 10.02 last 68.48) - open/close 1342 (time 1788836081.93 total 20.03 last 65.54) - open/close 2106 (time 1788836091.94 total 30.04 last 76.30) - open/close 3018 (time 1788836101.95 total 40.05 last 91.11) - open/close 3791 (time 1788836111.96 total 50.06 last 77.18) - open/close 4602 (time 1788836121.96 total 60.06 last 81.09) - open/close 5430 (time 1788836131.97 total 70.07 last 82.73) - open/close 6264 (time 1788836141.98 total 80.08 last 83.37) - open/close 7003 (time 1788836151.99 total 90.09 last 73.80) - open/close 7814 (time 1788836162.01 total 100.11 last 80.93) - open/close 8700 (time 1788836172.01 total 110.11 last 88.58) - open/close 9574 (time 1788836182.02 total 120.12 last 87.32) total: 10000 open/close in 124.79 seconds: 80.14 ops/second llite.lustre-ffff9cc9461f4000.statahead_batch_max=0 llite.lustre-ffff9cc9461f4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.100s user 0m0.326s sys 0m13.505s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9cc9461f4000.statahead_batch_max=32 llite.lustre-ffff9cc9461f4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m36.162s user 0m0.305s sys 0m13.193s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats= snapshot_time: 1788836369.225369065 secs.nsecs start_time: 1788836262.735877523 secs.nsecs elapsed_time: 106.489491542 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=0 llite.lustre-ffff9cc9461f4000.statahead_max=2048 llite.lustre-ffff9cc9461f4000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=0 total: 100 open/close in 2.90 seconds: 34.53 ops/second llite.lustre-ffff9cc9461f4000.statahead_max=0 101 real 0m2.153s user 0m0.005s sys 0m0.319s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9cc9461f4000.statahead_max=2048 llite.lustre-ffff9cc9461f4000.statahead_max=2048 101 real 0m0.779s user 0m0.002s sys 0m0.151s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 667 (time 1788836693.46 total 10.01 last 66.62) total: 900 open/close in 14.08 seconds: 63.90 ops/second llite.lustre-ffff9cc9461f4000.statahead_max=0 1001 real 0m20.991s user 0m0.040s sys 0m2.888s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9cc9461f4000.statahead_max=2048 llite.lustre-ffff9cc9461f4000.statahead_max=2048 1001 real 0m7.315s user 0m0.036s sys 0m1.424s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9cc9461f4000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats= snapshot_time: 1788836759.506272912 secs.nsecs start_time: 1788836371.620741051 secs.nsecs elapsed_time: 387.885531861 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 525 (time 1788836772.85 total 10.00 last 52.48) - open/close 1153 (time 1788836782.85 total 20.01 last 62.77) - open/close 1902 (time 1788836792.86 total 30.02 last 74.83) - open/close 2629 (time 1788836802.87 total 40.02 last 72.66) - open/close 3294 (time 1788836812.89 total 50.05 last 66.35) - open/close 4008 (time 1788836822.90 total 60.05 last 71.34) - open/close 4838 (time 1788836832.90 total 70.06 last 82.98) - open/close 5494 (time 1788836842.91 total 80.06 last 65.57) - open/close 6180 (time 1788836852.91 total 90.06 last 68.58) - open/close 6959 (time 1788836862.91 total 100.07 last 77.86) - open/close 7805 (time 1788836872.93 total 110.08 last 84.49) - open/close 8663 (time 1788836882.93 total 120.09 last 85.74) - open/close 9459 (time 1788836892.94 total 130.09 last 79.56) total: 10000 open/close in 137.22 seconds: 72.87 ops/second llite.lustre-ffff9cc9461f4000.statahead_batch_max=0 llite.lustre-ffff9cc9461f4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.927s user 0m0.305s sys 0m13.924s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9cc9461f4000.statahead_batch_max=256 llite.lustre-ffff9cc9461f4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.816s user 0m0.349s sys 0m13.801s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9cc9461f4000.batch_stats= snapshot_time: 1788837030.889080008 secs.nsecs start_time: 1788836971.613721039 secs.nsecs elapsed_time: 59.275358969 secs.nsecs subreqs per batch batches %% cum %% 1: 122 47 47 2: 37 14 61 4: 17 6 68 8: 13 5 73 16: 9 3 77 32: 19 7 84 64: 5 1 86 128: 0 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9cc9461f4000.statahead_batch_max=64 llite.lustre-ffff9cc9461f4000.statahead_max=128