== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:29:11 (1790746151) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=0 llite.lustre-ffff95e57a7b5000.statahead_max=32 llite.lustre-ffff95e57a7b5000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=0 total: 100 open/close in 2.59 seconds: 38.65 ops/second llite.lustre-ffff95e57a7b5000.statahead_max=0 101 real 0m2.155s user 0m0.011s sys 0m0.345s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95e57a7b5000.statahead_max=32 llite.lustre-ffff95e57a7b5000.statahead_max=32 101 real 0m1.070s user 0m0.011s sys 0m0.182s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 30 fname_total: 101 hit_total: 4381 miss_total: 771 swrong=2 ewrong=2 - open/close 724 (time 1790746179.21 total 10.00 last 72.40) total: 900 open/close in 12.52 seconds: 71.88 ops/second llite.lustre-ffff95e57a7b5000.statahead_max=0 1001 real 0m22.002s user 0m0.043s sys 0m3.000s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff95e57a7b5000.statahead_max=32 llite.lustre-ffff95e57a7b5000.statahead_max=32 1001 real 0m12.193s user 0m0.030s sys 0m1.510s 'ls -l' 1000 files with statahead: 12 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 101 hit_total: 5377 miss_total: 775 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff95e57a7b5000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 101 hit_total: 5377 miss_total: 775 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats= snapshot_time: 1790746255.012210175 secs.nsecs start_time: 1790746156.218048331 secs.nsecs elapsed_time: 98.794161844 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 643 (time 1790746267.79 total 10.00 last 64.30) - open/close 1250 (time 1790746277.80 total 20.01 last 60.62) - open/close 1912 (time 1790746287.81 total 30.02 last 66.19) - open/close 2573 (time 1790746297.81 total 40.02 last 66.09) - open/close 3250 (time 1790746307.81 total 50.02 last 67.69) - open/close 3865 (time 1790746317.81 total 60.02 last 61.49) - open/close 4528 (time 1790746327.81 total 70.03 last 66.27) - open/close 5150 (time 1790746337.82 total 80.04 last 62.14) - open/close 5768 (time 1790746347.84 total 90.05 last 61.73) - open/close 6337 (time 1790746357.84 total 100.05 last 56.89) - open/close 6997 (time 1790746367.84 total 110.05 last 65.97) - open/close 7652 (time 1790746377.85 total 120.06 last 65.46) - open/close 8202 (time 1790746387.85 total 130.06 last 54.98) - open/close 8860 (time 1790746397.85 total 140.06 last 65.79) - open/close 9441 (time 1790746407.86 total 150.07 last 58.05) total: 10000 open/close in 159.04 seconds: 62.88 ops/second llite.lustre-ffff95e57a7b5000.statahead_batch_max=0 llite.lustre-ffff95e57a7b5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.480s user 0m0.320s sys 0m15.410s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95e57a7b5000.statahead_batch_max=32 llite.lustre-ffff95e57a7b5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m43.792s user 0m0.301s sys 0m14.263s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats= snapshot_time: 1790746604.752846363 secs.nsecs start_time: 1790746488.242695379 secs.nsecs elapsed_time: 116.510150984 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=0 llite.lustre-ffff95e57a7b5000.statahead_max=2048 llite.lustre-ffff95e57a7b5000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=0 total: 100 open/close in 2.95 seconds: 33.95 ops/second llite.lustre-ffff95e57a7b5000.statahead_max=0 101 real 0m2.445s user 0m0.006s sys 0m0.343s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95e57a7b5000.statahead_max=2048 llite.lustre-ffff95e57a7b5000.statahead_max=2048 101 real 0m0.888s user 0m0.005s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 667 (time 1790746926.77 total 10.01 last 66.66) total: 900 open/close in 13.89 seconds: 64.80 ops/second llite.lustre-ffff95e57a7b5000.statahead_max=0 1001 real 0m22.602s user 0m0.035s sys 0m3.093s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff95e57a7b5000.statahead_max=2048 llite.lustre-ffff95e57a7b5000.statahead_max=2048 1001 real 0m7.266s user 0m0.025s sys 0m1.668s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff95e57a7b5000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats= snapshot_time: 1790746994.606653672 secs.nsecs start_time: 1790746607.205453643 secs.nsecs elapsed_time: 387.401200029 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 682 (time 1790747006.88 total 10.00 last 68.18) - open/close 1382 (time 1790747016.88 total 20.01 last 69.94) - open/close 2086 (time 1790747026.89 total 30.01 last 70.39) - open/close 2731 (time 1790747036.91 total 40.03 last 64.37) - open/close 3358 (time 1790747046.92 total 50.05 last 62.62) - open/close 4022 (time 1790747056.93 total 60.06 last 66.35) - open/close 4640 (time 1790747066.94 total 70.07 last 61.74) - open/close 5255 (time 1790747076.95 total 80.07 last 61.45) - open/close 5915 (time 1790747086.95 total 90.07 last 65.99) - open/close 6590 (time 1790747096.95 total 100.08 last 67.47) - open/close 7295 (time 1790747106.97 total 110.10 last 70.37) - open/close 8001 (time 1790747116.98 total 120.11 last 70.50) - open/close 8698 (time 1790747126.99 total 130.12 last 69.65) - open/close 9400 (time 1790747137.00 total 140.13 last 70.13) total: 10000 open/close in 148.33 seconds: 67.42 ops/second llite.lustre-ffff95e57a7b5000.statahead_batch_max=0 llite.lustre-ffff95e57a7b5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.690s user 0m0.314s sys 0m14.581s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95e57a7b5000.statahead_batch_max=256 llite.lustre-ffff95e57a7b5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.701s user 0m0.277s sys 0m15.635s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95e57a7b5000.batch_stats= snapshot_time: 1790747273.023776107 secs.nsecs start_time: 1790747215.422541211 secs.nsecs elapsed_time: 57.601234896 secs.nsecs subreqs per batch batches %% cum %% 1: 146 55 55 2: 26 9 64 4: 16 6 70 8: 11 4 75 16: 8 3 78 32: 20 7 85 64: 2 0 86 128: 1 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95e57a7b5000.statahead_batch_max=64 llite.lustre-ffff95e57a7b5000.statahead_max=128