== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:53:40 (1787554420) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=0 llite.lustre-ffff89ff03967000.statahead_max=32 llite.lustre-ffff89ff03967000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.84 seconds: 35.26 ops/second llite.lustre-ffff89ff03967000.statahead_max=0 101 real 0m2.127s user 0m0.006s sys 0m0.327s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff89ff03967000.statahead_max=32 llite.lustre-ffff89ff03967000.statahead_max=32 101 real 0m0.915s user 0m0.006s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 109 statahead wrong: 2 agl total: 103 list_total: 31 fname_total: 64 hit_total: 3612 miss_total: 1743 swrong=2 ewrong=2 - open/close 675 (time 1787554447.34 total 10.00 last 67.47) total: 900 open/close in 12.83 seconds: 70.13 ops/second llite.lustre-ffff89ff03967000.statahead_max=0 1001 real 0m23.558s user 0m0.037s sys 0m3.020s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff89ff03967000.statahead_max=32 llite.lustre-ffff89ff03967000.statahead_max=32 1001 real 0m8.756s user 0m0.028s sys 0m1.421s 'ls -l' 1000 files with statahead: 9 sec statahead total: 110 statahead wrong: 2 agl total: 104 list_total: 32 fname_total: 64 hit_total: 4609 miss_total: 1746 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff89ff03967000.statahead_stats= statahead total: 110 statahead wrong: 2 agl total: 104 list_total: 32 fname_total: 64 hit_total: 4609 miss_total: 1746 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787554516.841691748 secs.nsecs start_time: 1787554425.018795489 secs.nsecs elapsed_time: 91.822896259 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 3 7 12 8: 3 7 20 16: 20 50 70 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787554516.841765080 secs.nsecs start_time: 1787554425.018958267 secs.nsecs elapsed_time: 91.822806813 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 3 7 12 8: 1 2 15 16: 24 60 75 32: 10 25 100 - open/close 844 (time 1787554529.44 total 10.01 last 84.34) - open/close 1590 (time 1787554539.45 total 20.01 last 74.56) - open/close 2348 (time 1787554549.46 total 30.02 last 75.73) - open/close 3178 (time 1787554559.47 total 40.04 last 82.88) - open/close 3994 (time 1787554569.48 total 50.04 last 81.55) - open/close 4770 (time 1787554579.48 total 60.04 last 77.60) - open/close 5587 (time 1787554589.49 total 70.05 last 81.64) - open/close 6186 (time 1787554599.50 total 80.06 last 59.83) - open/close 6946 (time 1787554609.50 total 90.06 last 75.99) - open/close 7647 (time 1787554619.50 total 100.06 last 70.10) - open/close 8453 (time 1787554629.51 total 110.08 last 80.51) - open/close 9321 (time 1787554639.52 total 120.08 last 86.72) total: 10000 open/close in 128.21 seconds: 78.00 ops/second llite.lustre-ffff89ff03967000.statahead_batch_max=0 llite.lustre-ffff89ff03967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.953s user 0m0.316s sys 0m15.638s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff89ff03967000.statahead_batch_max=32 llite.lustre-ffff89ff03967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m33.577s user 0m0.284s sys 0m14.410s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787554818.607354062 secs.nsecs start_time: 1787554716.815549174 secs.nsecs elapsed_time: 101.791804888 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787554818.607428332 secs.nsecs start_time: 1787554716.815995754 secs.nsecs elapsed_time: 101.791432578 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=0 llite.lustre-ffff89ff03967000.statahead_max=2048 llite.lustre-ffff89ff03967000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.82 seconds: 35.44 ops/second llite.lustre-ffff89ff03967000.statahead_max=0 101 real 0m2.159s user 0m0.006s sys 0m0.308s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff89ff03967000.statahead_max=2048 llite.lustre-ffff89ff03967000.statahead_max=2048 101 real 0m0.996s user 0m0.003s sys 0m0.208s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 866 (time 1787555114.92 total 10.00 last 86.60) total: 900 open/close in 10.51 seconds: 85.63 ops/second llite.lustre-ffff89ff03967000.statahead_max=0 1001 real 0m21.330s user 0m0.034s sys 0m3.006s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff89ff03967000.statahead_max=2048 llite.lustre-ffff89ff03967000.statahead_max=2048 1001 real 0m5.961s user 0m0.031s sys 0m1.509s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff89ff03967000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787555172.537501057 secs.nsecs start_time: 1787554822.364807703 secs.nsecs elapsed_time: 350.172693354 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 3 37 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787555172.537589663 secs.nsecs start_time: 1787554822.365018603 secs.nsecs elapsed_time: 350.172571060 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 2 25 37 8: 1 12 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 800 (time 1787555185.20 total 10.01 last 79.94) - open/close 1669 (time 1787555195.20 total 20.01 last 86.89) - open/close 2479 (time 1787555205.20 total 30.01 last 80.99) - open/close 3258 (time 1787555215.20 total 40.01 last 77.88) - open/close 3992 (time 1787555225.22 total 50.03 last 73.26) - open/close 4844 (time 1787555235.23 total 60.04 last 85.15) - open/close 5636 (time 1787555245.23 total 70.04 last 79.18) - open/close 6454 (time 1787555255.24 total 80.05 last 81.73) - open/close 7317 (time 1787555265.25 total 90.06 last 86.22) - open/close 8196 (time 1787555275.25 total 100.06 last 87.85) - open/close 9091 (time 1787555285.26 total 110.07 last 89.48) - open/close 9984 (time 1787555295.27 total 120.08 last 89.21) total: 10000 open/close in 120.22 seconds: 83.18 ops/second llite.lustre-ffff89ff03967000.statahead_batch_max=0 llite.lustre-ffff89ff03967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.214s user 0m0.311s sys 0m15.123s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff89ff03967000.statahead_batch_max=256 llite.lustre-ffff89ff03967000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff89ff03967000.stats=clear mdc.lustre-MDT0001-mdc-ffff89ff03967000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.218s user 0m0.288s sys 0m15.264s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787555428.120984366 secs.nsecs start_time: 1787555366.814152252 secs.nsecs elapsed_time: 61.306832114 secs.nsecs subreqs per batch batches %% cum %% 1: 116 44 44 2: 38 14 59 4: 30 11 70 8: 11 4 75 16: 5 1 76 32: 19 7 84 64: 6 2 86 128: 1 0 86 256: 34 13 100 mdc.lustre-MDT0001-mdc-ffff89ff03967000.batch_stats= snapshot_time: 1787555428.121066083 secs.nsecs start_time: 1787555366.814222287 secs.nsecs elapsed_time: 61.306843796 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff89ff03967000.statahead_batch_max=64 llite.lustre-ffff89ff03967000.statahead_max=128