== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:53:55 (1790733235) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=0 llite.lustre-ffff99c409960000.statahead_max=32 llite.lustre-ffff99c409960000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=0 total: 100 open/close in 2.67 seconds: 37.48 ops/second llite.lustre-ffff99c409960000.statahead_max=0 101 real 0m2.189s user 0m0.004s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff99c409960000.statahead_max=32 llite.lustre-ffff99c409960000.statahead_max=32 101 real 0m0.962s user 0m0.008s sys 0m0.144s 'ls -l' 100 files with statahead: 1 sec statahead total: 120 statahead wrong: 2 agl total: 114 list_total: 29 fname_total: 69 hit_total: 4151 miss_total: 1274 swrong=2 ewrong=2 - open/close 734 (time 1790733262.06 total 10.01 last 73.33) total: 900 open/close in 12.01 seconds: 74.95 ops/second llite.lustre-ffff99c409960000.statahead_max=0 1001 real 0m20.416s user 0m0.033s sys 0m2.867s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff99c409960000.statahead_max=32 llite.lustre-ffff99c409960000.statahead_max=32 1001 real 0m10.049s user 0m0.032s sys 0m1.343s 'ls -l' 1000 files with statahead: 10 sec statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 69 hit_total: 5148 miss_total: 1277 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff99c409960000.statahead_stats= statahead total: 121 statahead wrong: 2 agl total: 115 list_total: 30 fname_total: 69 hit_total: 5148 miss_total: 1277 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats= snapshot_time: 1790733333.145074750 secs.nsecs start_time: 1790733239.749314667 secs.nsecs elapsed_time: 93.395760083 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 643 (time 1790733346.25 total 10.01 last 64.26) - open/close 1300 (time 1790733356.27 total 20.02 last 65.58) - open/close 1960 (time 1790733366.28 total 30.04 last 65.93) - open/close 2765 (time 1790733376.29 total 40.04 last 80.47) - open/close 3579 (time 1790733386.30 total 50.05 last 81.31) - open/close 4373 (time 1790733396.31 total 60.06 last 79.33) - open/close 5098 (time 1790733406.31 total 70.06 last 72.47) - open/close 5727 (time 1790733416.32 total 80.08 last 62.82) - open/close 6349 (time 1790733426.35 total 90.10 last 62.06) - open/close 6940 (time 1790733436.35 total 100.10 last 59.08) - open/close 7598 (time 1790733446.35 total 110.11 last 65.78) - open/close 8317 (time 1790733456.36 total 120.11 last 71.84) - open/close 8913 (time 1790733466.37 total 130.13 last 59.52) - open/close 9583 (time 1790733476.43 total 140.18 last 66.66) total: 10000 open/close in 146.04 seconds: 68.48 ops/second llite.lustre-ffff99c409960000.statahead_batch_max=0 llite.lustre-ffff99c409960000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.521s user 0m0.332s sys 0m14.294s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff99c409960000.statahead_batch_max=32 llite.lustre-ffff99c409960000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.755s user 0m0.302s sys 0m13.661s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats= snapshot_time: 1790733677.253365043 secs.nsecs start_time: 1790733552.627427488 secs.nsecs elapsed_time: 124.625937555 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=0 llite.lustre-ffff99c409960000.statahead_max=2048 llite.lustre-ffff99c409960000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=0 total: 100 open/close in 2.17 seconds: 46.09 ops/second llite.lustre-ffff99c409960000.statahead_max=0 101 real 0m2.693s user 0m0.005s sys 0m0.322s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff99c409960000.statahead_max=2048 llite.lustre-ffff99c409960000.statahead_max=2048 101 real 0m1.098s user 0m0.003s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 735 (time 1790734002.42 total 10.01 last 73.39) total: 900 open/close in 12.09 seconds: 74.42 ops/second llite.lustre-ffff99c409960000.statahead_max=0 1001 real 0m19.857s user 0m0.028s sys 0m2.950s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff99c409960000.statahead_max=2048 llite.lustre-ffff99c409960000.statahead_max=2048 1001 real 0m7.940s user 0m0.028s sys 0m1.265s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff99c409960000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats= snapshot_time: 1790734069.930968692 secs.nsecs start_time: 1790733681.029650854 secs.nsecs elapsed_time: 388.901317838 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 835 (time 1790734082.69 total 10.02 last 83.34) - open/close 1566 (time 1790734092.70 total 20.03 last 73.04) - open/close 2280 (time 1790734102.71 total 30.04 last 71.33) - open/close 2948 (time 1790734112.73 total 40.06 last 66.66) - open/close 3718 (time 1790734122.74 total 50.06 last 76.96) - open/close 4477 (time 1790734132.74 total 60.07 last 75.89) - open/close 5227 (time 1790734142.75 total 70.07 last 74.94) - open/close 5972 (time 1790734152.75 total 80.08 last 74.44) - open/close 6642 (time 1790734162.77 total 90.10 last 66.92) - open/close 7344 (time 1790734172.78 total 100.11 last 70.09) - open/close 8068 (time 1790734182.79 total 110.12 last 72.36) - open/close 8772 (time 1790734192.79 total 120.12 last 70.38) - open/close 9554 (time 1790734202.81 total 130.14 last 78.03) total: 10000 open/close in 135.58 seconds: 73.75 ops/second llite.lustre-ffff99c409960000.statahead_batch_max=0 llite.lustre-ffff99c409960000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.108s user 0m0.311s sys 0m14.063s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff99c409960000.statahead_batch_max=256 llite.lustre-ffff99c409960000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff99c409960000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.035s user 0m0.305s sys 0m13.935s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff99c409960000.batch_stats= snapshot_time: 1790734330.996341337 secs.nsecs start_time: 1790734269.992394077 secs.nsecs elapsed_time: 61.003947260 secs.nsecs subreqs per batch batches %% cum %% 1: 59 32 32 2: 32 17 49 4: 15 8 57 8: 12 6 64 16: 7 3 68 32: 21 11 79 64: 1 0 80 128: 0 0 80 256: 36 19 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff99c409960000.statahead_batch_max=64 llite.lustre-ffff99c409960000.statahead_max=128