== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:42:05 (1789126925) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=0 llite.lustre-ffff88ae896d0000.statahead_max=32 llite.lustre-ffff88ae896d0000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=0 total: 100 open/close in 2.61 seconds: 38.27 ops/second llite.lustre-ffff88ae896d0000.statahead_max=0 101 real 0m2.384s user 0m0.005s sys 0m0.322s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff88ae896d0000.statahead_max=32 llite.lustre-ffff88ae896d0000.statahead_max=32 101 real 0m1.291s user 0m0.005s sys 0m0.150s 'ls -l' 100 files with statahead: 1 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 29 fname_total: 66 hit_total: 4033 miss_total: 1650 swrong=2 ewrong=2 - open/close 608 (time 1789126954.06 total 10.03 last 60.62) total: 900 open/close in 13.73 seconds: 65.56 ops/second llite.lustre-ffff88ae896d0000.statahead_max=0 1001 real 0m19.414s user 0m0.036s sys 0m2.802s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff88ae896d0000.statahead_max=32 llite.lustre-ffff88ae896d0000.statahead_max=32 1001 real 0m15.681s user 0m0.028s sys 0m1.374s 'ls -l' 1000 files with statahead: 16 sec statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 30 fname_total: 66 hit_total: 5028 miss_total: 1655 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff88ae896d0000.statahead_stats= statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 30 fname_total: 66 hit_total: 5028 miss_total: 1655 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats= snapshot_time: 1789127033.597563011 secs.nsecs start_time: 1789126930.699658968 secs.nsecs elapsed_time: 102.897904043 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 770 (time 1789127046.34 total 10.01 last 76.90) - open/close 1486 (time 1789127056.34 total 20.02 last 71.56) - open/close 2141 (time 1789127066.35 total 30.02 last 65.47) - open/close 2921 (time 1789127076.35 total 40.03 last 77.96) - open/close 3734 (time 1789127086.36 total 50.03 last 81.27) - open/close 4475 (time 1789127096.36 total 60.04 last 74.06) - open/close 5265 (time 1789127106.37 total 70.04 last 78.95) - open/close 6035 (time 1789127116.37 total 80.05 last 76.99) - open/close 6753 (time 1789127126.38 total 90.05 last 71.74) - open/close 7421 (time 1789127136.39 total 100.07 last 66.70) - open/close 8227 (time 1789127146.39 total 110.07 last 80.59) - open/close 8986 (time 1789127156.41 total 120.09 last 75.77) - open/close 9761 (time 1789127166.42 total 130.09 last 77.44) total: 10000 open/close in 132.99 seconds: 75.19 ops/second llite.lustre-ffff88ae896d0000.statahead_batch_max=0 llite.lustre-ffff88ae896d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.554s user 0m0.320s sys 0m13.684s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88ae896d0000.statahead_batch_max=32 llite.lustre-ffff88ae896d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.531s user 0m0.303s sys 0m12.505s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats= snapshot_time: 1789127372.116641480 secs.nsecs start_time: 1789127254.458683867 secs.nsecs elapsed_time: 117.657957613 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=0 llite.lustre-ffff88ae896d0000.statahead_max=2048 llite.lustre-ffff88ae896d0000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=0 total: 100 open/close in 2.98 seconds: 33.51 ops/second llite.lustre-ffff88ae896d0000.statahead_max=0 101 real 0m1.611s user 0m0.007s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff88ae896d0000.statahead_max=2048 llite.lustre-ffff88ae896d0000.statahead_max=2048 101 real 0m1.658s user 0m0.008s sys 0m0.176s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 690 (time 1789127709.84 total 10.01 last 68.93) total: 900 open/close in 13.40 seconds: 67.15 ops/second llite.lustre-ffff88ae896d0000.statahead_max=0 1001 real 0m23.370s user 0m0.032s sys 0m2.938s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff88ae896d0000.statahead_max=2048 llite.lustre-ffff88ae896d0000.statahead_max=2048 1001 real 0m6.519s user 0m0.032s sys 0m1.200s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff88ae896d0000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats= snapshot_time: 1789127782.414650404 secs.nsecs start_time: 1789127374.813795987 secs.nsecs elapsed_time: 407.600854417 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 616 (time 1789127795.61 total 10.02 last 61.48) - open/close 1332 (time 1789127805.62 total 20.03 last 71.54) - open/close 2115 (time 1789127815.62 total 30.03 last 78.27) - open/close 2721 (time 1789127825.65 total 40.06 last 60.43) - open/close 3376 (time 1789127835.66 total 50.07 last 65.43) - open/close 4040 (time 1789127845.67 total 60.08 last 66.36) - open/close 4860 (time 1789127855.69 total 70.10 last 81.83) - open/close 5534 (time 1789127865.70 total 80.11 last 67.33) - open/close 6156 (time 1789127875.70 total 90.11 last 62.16) - open/close 6864 (time 1789127885.72 total 100.13 last 70.71) - open/close 7566 (time 1789127895.72 total 110.14 last 70.15) - open/close 8120 (time 1789127905.75 total 120.16 last 55.28) - open/close 8666 (time 1789127915.76 total 130.17 last 54.52) - open/close 9244 (time 1789127925.76 total 140.17 last 57.79) - open/close 9698 (time 1789127935.79 total 150.20 last 45.28) total: 10000 open/close in 157.91 seconds: 63.33 ops/second llite.lustre-ffff88ae896d0000.statahead_batch_max=0 llite.lustre-ffff88ae896d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.081s user 0m0.354s sys 0m13.903s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff88ae896d0000.statahead_batch_max=256 llite.lustre-ffff88ae896d0000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff88ae896d0000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.473s user 0m0.333s sys 0m13.208s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff88ae896d0000.batch_stats= snapshot_time: 1789128099.124796979 secs.nsecs start_time: 1789128024.448277026 secs.nsecs elapsed_time: 74.676519953 secs.nsecs subreqs per batch batches %% cum %% 1: 91 36 36 2: 37 14 50 4: 28 11 62 8: 28 11 73 16: 16 6 79 32: 13 5 84 64: 3 1 86 128: 0 0 86 256: 35 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff88ae896d0000.statahead_batch_max=64 llite.lustre-ffff88ae896d0000.statahead_max=128