== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:58:27 (1789027107) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=0 llite.lustre-ffff9c16c8ef4000.statahead_max=32 llite.lustre-ffff9c16c8ef4000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.39 ops/second llite.lustre-ffff9c16c8ef4000.statahead_max=0 101 real 0m2.488s user 0m0.004s sys 0m0.368s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c16c8ef4000.statahead_max=32 llite.lustre-ffff9c16c8ef4000.statahead_max=32 101 real 0m1.077s user 0m0.006s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 29 fname_total: 96 hit_total: 3596 miss_total: 1739 swrong=2 ewrong=2 - open/close 679 (time 1789027134.64 total 10.01 last 67.86) total: 900 open/close in 13.27 seconds: 67.82 ops/second llite.lustre-ffff9c16c8ef4000.statahead_max=0 1001 real 0m28.124s user 0m0.037s sys 0m3.168s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffff9c16c8ef4000.statahead_max=32 llite.lustre-ffff9c16c8ef4000.statahead_max=32 1001 real 0m11.134s user 0m0.043s sys 0m1.459s 'ls -l' 1000 files with statahead: 11 sec statahead total: 128 statahead wrong: 2 agl total: 123 list_total: 30 fname_total: 96 hit_total: 4594 miss_total: 1741 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff9c16c8ef4000.statahead_stats= statahead total: 128 statahead wrong: 2 agl total: 123 list_total: 30 fname_total: 96 hit_total: 4594 miss_total: 1741 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats= snapshot_time: 1789027210.344584127 secs.nsecs start_time: 1789027111.948178251 secs.nsecs elapsed_time: 98.396405876 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 683 (time 1789027222.51 total 10.00 last 68.28) - open/close 1357 (time 1789027232.52 total 20.01 last 67.34) - open/close 2015 (time 1789027242.54 total 30.03 last 65.69) - open/close 2673 (time 1789027252.54 total 40.03 last 65.80) - open/close 3269 (time 1789027262.54 total 50.03 last 59.59) - open/close 3909 (time 1789027272.56 total 60.05 last 63.91) - open/close 4573 (time 1789027282.56 total 70.05 last 66.37) - open/close 5177 (time 1789027292.56 total 80.05 last 60.38) - open/close 5819 (time 1789027302.57 total 90.06 last 64.16) - open/close 6455 (time 1789027312.57 total 100.06 last 63.59) - open/close 7131 (time 1789027322.59 total 110.08 last 67.46) - open/close 7781 (time 1789027332.59 total 120.08 last 64.98) - open/close 8398 (time 1789027342.61 total 130.10 last 61.63) - open/close 9088 (time 1789027352.61 total 140.10 last 68.96) - open/close 9678 (time 1789027362.62 total 150.11 last 58.92) total: 10000 open/close in 155.27 seconds: 64.40 ops/second llite.lustre-ffff9c16c8ef4000.statahead_batch_max=0 llite.lustre-ffff9c16c8ef4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.347s user 0m0.322s sys 0m14.736s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c16c8ef4000.statahead_batch_max=32 llite.lustre-ffff9c16c8ef4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.506s user 0m0.274s sys 0m13.895s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats= snapshot_time: 1789027556.117947172 secs.nsecs start_time: 1789027444.458376902 secs.nsecs elapsed_time: 111.659570270 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=0 llite.lustre-ffff9c16c8ef4000.statahead_max=2048 llite.lustre-ffff9c16c8ef4000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=0 total: 100 open/close in 3.34 seconds: 29.95 ops/second llite.lustre-ffff9c16c8ef4000.statahead_max=0 101 real 0m2.737s user 0m0.008s sys 0m0.357s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9c16c8ef4000.statahead_max=2048 llite.lustre-ffff9c16c8ef4000.statahead_max=2048 101 real 0m0.788s user 0m0.004s sys 0m0.202s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 668 (time 1789027897.65 total 10.01 last 66.71) total: 900 open/close in 13.65 seconds: 65.94 ops/second llite.lustre-ffff9c16c8ef4000.statahead_max=0 1001 real 0m23.406s user 0m0.038s sys 0m2.945s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9c16c8ef4000.statahead_max=2048 llite.lustre-ffff9c16c8ef4000.statahead_max=2048 1001 real 0m6.993s user 0m0.028s sys 0m1.446s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff9c16c8ef4000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats= snapshot_time: 1789027963.571280629 secs.nsecs start_time: 1789027559.212799510 secs.nsecs elapsed_time: 404.358481119 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 621 (time 1789027976.14 total 10.00 last 62.09) - open/close 1255 (time 1789027986.14 total 20.00 last 63.38) - open/close 1867 (time 1789027996.16 total 30.02 last 61.11) - open/close 2506 (time 1789028006.18 total 40.04 last 63.78) - open/close 3138 (time 1789028016.19 total 50.05 last 63.11) - open/close 3761 (time 1789028026.21 total 60.07 last 62.20) - open/close 4438 (time 1789028036.22 total 70.08 last 67.63) - open/close 5094 (time 1789028046.22 total 80.08 last 65.58) - open/close 5685 (time 1789028056.22 total 90.08 last 59.07) - open/close 6307 (time 1789028066.23 total 100.09 last 62.18) - open/close 6947 (time 1789028076.24 total 110.10 last 63.93) - open/close 7636 (time 1789028086.24 total 120.10 last 68.87) - open/close 8266 (time 1789028096.25 total 130.11 last 62.95) - open/close 8871 (time 1789028106.25 total 140.11 last 60.50) - open/close 9526 (time 1789028116.26 total 150.12 last 65.43) total: 10000 open/close in 158.69 seconds: 63.02 ops/second llite.lustre-ffff9c16c8ef4000.statahead_batch_max=0 llite.lustre-ffff9c16c8ef4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.134s user 0m0.348s sys 0m15.833s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9c16c8ef4000.statahead_batch_max=256 llite.lustre-ffff9c16c8ef4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.082s user 0m0.312s sys 0m14.479s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9c16c8ef4000.batch_stats= snapshot_time: 1789028269.364327360 secs.nsecs start_time: 1789028202.149414313 secs.nsecs elapsed_time: 67.214913047 secs.nsecs subreqs per batch batches %% cum %% 1: 76 42 42 2: 20 11 53 4: 7 3 57 8: 12 6 64 16: 5 2 67 32: 14 7 75 64: 5 2 78 128: 3 1 79 256: 36 20 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9c16c8ef4000.statahead_batch_max=64 llite.lustre-ffff9c16c8ef4000.statahead_max=128