== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:53:26 (1788656006) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=0 llite.lustre-ffff9b5490665800.statahead_max=32 llite.lustre-ffff9b5490665800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=0 total: 100 open/close in 3.60 seconds: 27.75 ops/second llite.lustre-ffff9b5490665800.statahead_max=0 101 real 0m2.980s user 0m0.004s sys 0m0.321s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b5490665800.statahead_max=32 llite.lustre-ffff9b5490665800.statahead_max=32 101 real 0m1.146s user 0m0.008s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 31 fname_total: 62 hit_total: 3729 miss_total: 1503 swrong=2 ewrong=2 - open/close 674 (time 1788656036.81 total 10.01 last 67.37) total: 900 open/close in 13.59 seconds: 66.23 ops/second llite.lustre-ffff9b5490665800.statahead_max=0 1001 real 0m22.569s user 0m0.032s sys 0m2.809s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9b5490665800.statahead_max=32 llite.lustre-ffff9b5490665800.statahead_max=32 1001 real 0m9.842s user 0m0.022s sys 0m1.357s 'ls -l' 1000 files with statahead: 10 sec statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 32 fname_total: 62 hit_total: 4724 miss_total: 1508 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9b5490665800.statahead_stats= statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 32 fname_total: 62 hit_total: 4724 miss_total: 1508 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats= snapshot_time: 1788656111.233750589 secs.nsecs start_time: 1788656011.843617810 secs.nsecs elapsed_time: 99.390132779 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 0 0 4 4: 0 0 4 8: 3 7 12 16: 2 4 17 32: 34 82 100 - open/close 597 (time 1788656124.44 total 10.01 last 59.66) - open/close 1219 (time 1788656134.44 total 20.01 last 62.16) - open/close 1881 (time 1788656144.44 total 30.01 last 66.20) - open/close 2525 (time 1788656154.45 total 40.02 last 64.38) - open/close 3173 (time 1788656164.46 total 50.03 last 64.72) - open/close 3819 (time 1788656174.47 total 60.03 last 64.56) - open/close 4492 (time 1788656184.48 total 70.05 last 67.21) - open/close 5141 (time 1788656194.48 total 80.05 last 64.87) - open/close 5789 (time 1788656204.48 total 90.05 last 64.79) - open/close 6432 (time 1788656214.49 total 100.06 last 64.28) - open/close 7046 (time 1788656224.50 total 110.07 last 61.30) - open/close 7659 (time 1788656234.51 total 120.07 last 61.29) - open/close 8248 (time 1788656244.52 total 130.09 last 58.79) - open/close 8793 (time 1788656254.54 total 140.11 last 54.41) - open/close 9383 (time 1788656264.55 total 150.12 last 58.96) total: 10000 open/close in 159.82 seconds: 62.57 ops/second llite.lustre-ffff9b5490665800.statahead_batch_max=0 llite.lustre-ffff9b5490665800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.282s user 0m0.290s sys 0m14.018s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b5490665800.statahead_batch_max=32 llite.lustre-ffff9b5490665800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m53.814s user 0m0.302s sys 0m13.239s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats= snapshot_time: 1788656475.358117070 secs.nsecs start_time: 1788656349.833377226 secs.nsecs elapsed_time: 125.524739844 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=0 llite.lustre-ffff9b5490665800.statahead_max=2048 llite.lustre-ffff9b5490665800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=0 total: 100 open/close in 2.69 seconds: 37.20 ops/second llite.lustre-ffff9b5490665800.statahead_max=0 101 real 0m2.689s user 0m0.003s sys 0m0.296s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b5490665800.statahead_max=2048 llite.lustre-ffff9b5490665800.statahead_max=2048 101 real 0m1.024s user 0m0.002s sys 0m0.157s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 663 (time 1788656839.93 total 10.00 last 66.29) total: 900 open/close in 14.18 seconds: 63.48 ops/second llite.lustre-ffff9b5490665800.statahead_max=0 1001 real 0m26.130s user 0m0.035s sys 0m3.246s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9b5490665800.statahead_max=2048 llite.lustre-ffff9b5490665800.statahead_max=2048 1001 real 0m8.466s user 0m0.028s sys 0m1.229s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9b5490665800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats= snapshot_time: 1788656917.318433488 secs.nsecs start_time: 1788656477.967513979 secs.nsecs elapsed_time: 439.350919509 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 606 (time 1788656930.09 total 10.01 last 60.54) - open/close 1241 (time 1788656940.09 total 20.01 last 63.47) - open/close 1801 (time 1788656950.10 total 30.02 last 55.97) - open/close 2306 (time 1788656960.12 total 40.04 last 50.40) - open/close 2808 (time 1788656970.13 total 50.05 last 50.17) - open/close 3421 (time 1788656980.14 total 60.06 last 61.23) - open/close 3951 (time 1788656990.14 total 70.06 last 52.99) - open/close 4564 (time 1788657000.15 total 80.06 last 61.27) - open/close 5067 (time 1788657010.15 total 90.07 last 50.26) - open/close 5587 (time 1788657020.16 total 100.08 last 51.96) - open/close 6124 (time 1788657030.16 total 110.08 last 53.69) - open/close 6716 (time 1788657040.16 total 120.08 last 59.19) - open/close 7343 (time 1788657050.17 total 130.09 last 62.64) - open/close 7945 (time 1788657060.20 total 140.12 last 60.06) - open/close 8574 (time 1788657070.22 total 150.14 last 62.74) - open/close 9201 (time 1788657080.23 total 160.15 last 62.67) - open/close 9828 (time 1788657090.23 total 170.15 last 62.70) total: 10000 open/close in 172.89 seconds: 57.84 ops/second llite.lustre-ffff9b5490665800.statahead_batch_max=0 llite.lustre-ffff9b5490665800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.133s user 0m0.341s sys 0m14.666s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b5490665800.statahead_batch_max=256 llite.lustre-ffff9b5490665800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b5490665800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.026s user 0m0.291s sys 0m13.699s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b5490665800.batch_stats= snapshot_time: 1788657236.773129122 secs.nsecs start_time: 1788657175.492421074 secs.nsecs elapsed_time: 61.280708048 secs.nsecs subreqs per batch batches %% cum %% 1: 94 44 44 2: 24 11 55 4: 14 6 62 8: 12 5 68 16: 8 3 72 32: 18 8 80 64: 6 2 83 128: 0 0 83 256: 35 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b5490665800.statahead_batch_max=64 llite.lustre-ffff9b5490665800.statahead_max=128