== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:39:33 (1788975573) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=0 llite.lustre-ffff88e39d55a000.statahead_max=32 llite.lustre-ffff88e39d55a000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.79 seconds: 35.81 ops/second llite.lustre-ffff88e39d55a000.statahead_max=0 101 real 0m2.187s user 0m0.004s sys 0m0.270s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff88e39d55a000.statahead_max=32 llite.lustre-ffff88e39d55a000.statahead_max=32 101 real 0m0.872s user 0m0.006s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 31 fname_total: 102 hit_total: 4504 miss_total: 1289 swrong=2 ewrong=2 - open/close 798 (time 1788975600.86 total 10.00 last 79.78) total: 900 open/close in 11.21 seconds: 80.31 ops/second llite.lustre-ffff88e39d55a000.statahead_max=0 1001 real 0m21.500s user 0m0.029s sys 0m2.874s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff88e39d55a000.statahead_max=32 llite.lustre-ffff88e39d55a000.statahead_max=32 1001 real 0m8.740s user 0m0.034s sys 0m1.422s 'ls -l' 1000 files with statahead: 9 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 32 fname_total: 102 hit_total: 5502 miss_total: 1291 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff88e39d55a000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 32 fname_total: 102 hit_total: 5502 miss_total: 1291 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788975662.754335706 secs.nsecs start_time: 1788975578.089480331 secs.nsecs elapsed_time: 84.664855375 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 3 7 12 8: 2 5 17 16: 19 47 65 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788975662.754399326 secs.nsecs start_time: 1788975578.090684404 secs.nsecs elapsed_time: 84.663714922 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 5 12 17 16: 18 46 64 32: 14 35 100 - open/close 830 (time 1788975675.73 total 10.00 last 82.98) - open/close 1517 (time 1788975685.74 total 20.01 last 68.63) - open/close 2077 (time 1788975695.76 total 30.03 last 55.91) - open/close 2717 (time 1788975705.77 total 40.04 last 63.95) - open/close 3453 (time 1788975715.79 total 50.06 last 73.41) - open/close 4183 (time 1788975725.79 total 60.07 last 72.98) - open/close 4983 (time 1788975735.80 total 70.08 last 79.93) - open/close 5833 (time 1788975745.80 total 80.08 last 84.99) - open/close 6779 (time 1788975755.82 total 90.09 last 94.47) - open/close 7697 (time 1788975765.84 total 100.11 last 91.64) - open/close 8637 (time 1788975775.84 total 110.11 last 93.97) - open/close 9421 (time 1788975785.84 total 120.11 last 78.40) total: 10000 open/close in 126.99 seconds: 78.75 ops/second llite.lustre-ffff88e39d55a000.statahead_batch_max=0 llite.lustre-ffff88e39d55a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.651s user 0m0.294s sys 0m15.933s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88e39d55a000.statahead_batch_max=32 llite.lustre-ffff88e39d55a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m31.814s user 0m0.279s sys 0m13.800s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788975950.388238746 secs.nsecs start_time: 1788975849.889906088 secs.nsecs elapsed_time: 100.498332658 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788975950.388304109 secs.nsecs start_time: 1788975849.889991439 secs.nsecs elapsed_time: 100.498312670 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=0 llite.lustre-ffff88e39d55a000.statahead_max=2048 llite.lustre-ffff88e39d55a000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.33 seconds: 42.92 ops/second llite.lustre-ffff88e39d55a000.statahead_max=0 101 real 0m1.795s user 0m0.002s sys 0m0.290s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff88e39d55a000.statahead_max=2048 llite.lustre-ffff88e39d55a000.statahead_max=2048 101 real 0m0.811s user 0m0.002s sys 0m0.193s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 765 (time 1788976227.90 total 10.01 last 76.46) total: 900 open/close in 11.65 seconds: 77.24 ops/second llite.lustre-ffff88e39d55a000.statahead_max=0 1001 real 0m19.902s user 0m0.030s sys 0m2.847s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff88e39d55a000.statahead_max=2048 llite.lustre-ffff88e39d55a000.statahead_max=2048 1001 real 0m6.472s user 0m0.033s sys 0m1.481s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff88e39d55a000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788976283.501224022 secs.nsecs start_time: 1788975952.288083070 secs.nsecs elapsed_time: 331.213140952 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 11 11 4: 1 11 22 8: 4 44 66 16: 0 0 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788976283.501286270 secs.nsecs start_time: 1788975952.288237820 secs.nsecs elapsed_time: 331.213048450 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 33 33 8: 1 11 44 16: 2 22 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 - open/close 725 (time 1788976295.43 total 10.00 last 72.47) - open/close 1612 (time 1788976305.44 total 20.01 last 88.66) - open/close 2451 (time 1788976315.45 total 30.02 last 83.83) - open/close 3280 (time 1788976325.45 total 40.02 last 82.89) - open/close 4142 (time 1788976335.45 total 50.02 last 86.17) - open/close 5070 (time 1788976345.46 total 60.03 last 92.75) - open/close 5913 (time 1788976355.47 total 70.04 last 84.22) - open/close 6756 (time 1788976365.47 total 80.04 last 84.25) - open/close 7628 (time 1788976375.49 total 90.06 last 87.06) - open/close 8357 (time 1788976385.49 total 100.06 last 72.89) - open/close 9219 (time 1788976395.50 total 110.07 last 86.10) total: 10000 open/close in 118.11 seconds: 84.66 ops/second llite.lustre-ffff88e39d55a000.statahead_batch_max=0 llite.lustre-ffff88e39d55a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.861s user 0m0.303s sys 0m14.866s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88e39d55a000.statahead_batch_max=256 llite.lustre-ffff88e39d55a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88e39d55a000.stats=clear mdc.lustre-MDT0001-mdc-ffff88e39d55a000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.937s user 0m0.308s sys 0m15.565s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788976529.005959503 secs.nsecs start_time: 1788976470.587907356 secs.nsecs elapsed_time: 58.418052147 secs.nsecs subreqs per batch batches %% cum %% 1: 145 48 48 2: 39 13 61 4: 28 9 71 8: 15 5 76 16: 12 4 80 32: 15 5 85 64: 9 3 88 128: 0 0 88 256: 34 11 100 mdc.lustre-MDT0001-mdc-ffff88e39d55a000.batch_stats= snapshot_time: 1788976529.006019129 secs.nsecs start_time: 1788976470.588061303 secs.nsecs elapsed_time: 58.417957826 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88e39d55a000.statahead_batch_max=64 llite.lustre-ffff88e39d55a000.statahead_max=128