== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:48:07 (1783968487) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=0 llite.lustre-ffff92cf4a560000.statahead_max=32 llite.lustre-ffff92cf4a560000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=0 total: 100 open/close in 2.18 seconds: 45.81 ops/second llite.lustre-ffff92cf4a560000.statahead_max=0 101 real 0m1.927s user 0m0.008s sys 0m0.329s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff92cf4a560000.statahead_max=32 llite.lustre-ffff92cf4a560000.statahead_max=32 101 real 0m1.263s user 0m0.006s sys 0m0.161s 'ls -l' 100 files with statahead: 2 sec statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 44 fname_total: 70 hit_total: 4028 miss_total: 1722 swrong=2 ewrong=2 - open/close 755 (time 1783968516.25 total 10.01 last 75.44) total: 900 open/close in 11.86 seconds: 75.86 ops/second llite.lustre-ffff92cf4a560000.statahead_max=0 1001 real 0m22.476s user 0m0.033s sys 0m3.217s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff92cf4a560000.statahead_max=32 llite.lustre-ffff92cf4a560000.statahead_max=32 1001 real 0m10.919s user 0m0.041s sys 0m1.505s 'ls -l' 1000 files with statahead: 11 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 45 fname_total: 70 hit_total: 5027 miss_total: 1723 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff92cf4a560000.statahead_stats= statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 45 fname_total: 70 hit_total: 5027 miss_total: 1723 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats= snapshot_time: 1783968591.907946052 secs.nsecs start_time: 1783968492.356631418 secs.nsecs elapsed_time: 99.551314634 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 803 (time 1783968604.50 total 10.01 last 80.25) - open/close 1669 (time 1783968614.51 total 20.01 last 86.53) - open/close 2595 (time 1783968624.51 total 30.02 last 92.59) - open/close 3466 (time 1783968634.52 total 40.02 last 87.06) - open/close 4318 (time 1783968644.54 total 50.04 last 85.05) - open/close 5200 (time 1783968654.54 total 60.04 last 88.17) - open/close 6032 (time 1783968664.54 total 70.04 last 83.19) - open/close 6884 (time 1783968674.54 total 80.05 last 85.17) - open/close 7395 (time 1783968684.55 total 90.06 last 51.04) - open/close 8216 (time 1783968694.56 total 100.06 last 82.03) - open/close 8978 (time 1783968704.57 total 110.07 last 76.12) - open/close 9746 (time 1783968714.58 total 120.08 last 76.77) total: 10000 open/close in 122.84 seconds: 81.40 ops/second llite.lustre-ffff92cf4a560000.statahead_batch_max=0 llite.lustre-ffff92cf4a560000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m15.224s user 0m0.333s sys 0m15.067s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92cf4a560000.statahead_batch_max=32 llite.lustre-ffff92cf4a560000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.565s user 0m0.322s sys 0m15.130s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats= snapshot_time: 1783968933.419382828 secs.nsecs start_time: 1783968804.198383998 secs.nsecs elapsed_time: 129.220998830 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=0 llite.lustre-ffff92cf4a560000.statahead_max=2048 llite.lustre-ffff92cf4a560000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=0 total: 100 open/close in 2.38 seconds: 42.10 ops/second llite.lustre-ffff92cf4a560000.statahead_max=0 101 real 0m2.482s user 0m0.005s sys 0m0.327s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92cf4a560000.statahead_max=2048 llite.lustre-ffff92cf4a560000.statahead_max=2048 101 real 0m0.721s user 0m0.004s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 792 (time 1783969244.31 total 10.01 last 79.12) total: 900 open/close in 11.64 seconds: 77.34 ops/second llite.lustre-ffff92cf4a560000.statahead_max=0 1001 real 0m19.124s user 0m0.039s sys 0m3.110s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff92cf4a560000.statahead_max=2048 llite.lustre-ffff92cf4a560000.statahead_max=2048 1001 real 0m8.703s user 0m0.023s sys 0m1.368s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff92cf4a560000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats= snapshot_time: 1783969312.698418952 secs.nsecs start_time: 1783968936.865564672 secs.nsecs elapsed_time: 375.832854280 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 877 (time 1783969325.15 total 10.00 last 87.69) - open/close 1657 (time 1783969335.15 total 20.00 last 77.99) - open/close 2439 (time 1783969345.16 total 30.01 last 78.18) - open/close 3367 (time 1783969355.17 total 40.02 last 92.71) - open/close 4169 (time 1783969365.17 total 50.02 last 80.18) - open/close 4945 (time 1783969375.18 total 60.03 last 77.54) - open/close 5732 (time 1783969385.18 total 70.03 last 78.67) - open/close 6587 (time 1783969395.20 total 80.05 last 85.35) - open/close 7337 (time 1783969405.20 total 90.05 last 74.98) - open/close 8168 (time 1783969415.21 total 100.06 last 83.01) - open/close 8844 (time 1783969425.22 total 110.07 last 67.52) - open/close 9584 (time 1783969435.23 total 120.07 last 73.98) total: 10000 open/close in 125.91 seconds: 79.42 ops/second llite.lustre-ffff92cf4a560000.statahead_batch_max=0 llite.lustre-ffff92cf4a560000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.022s user 0m0.289s sys 0m15.064s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92cf4a560000.statahead_batch_max=256 llite.lustre-ffff92cf4a560000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92cf4a560000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.036s user 0m0.313s sys 0m15.142s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92cf4a560000.batch_stats= snapshot_time: 1783969611.571488124 secs.nsecs start_time: 1783969524.972384234 secs.nsecs elapsed_time: 86.599103890 secs.nsecs subreqs per batch batches %% cum %% 1: 199 48 48 2: 76 18 66 4: 41 9 76 8: 23 5 82 16: 20 4 87 32: 16 3 91 64: 3 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92cf4a560000.statahead_batch_max=64 llite.lustre-ffff92cf4a560000.statahead_max=128