== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:16:41 (1788218201) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=0 llite.lustre-ffff8def10755000.statahead_max=32 llite.lustre-ffff8def10755000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=0 total: 100 open/close in 3.13 seconds: 31.99 ops/second llite.lustre-ffff8def10755000.statahead_max=0 101 real 0m3.020s user 0m0.003s sys 0m0.319s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8def10755000.statahead_max=32 llite.lustre-ffff8def10755000.statahead_max=32 101 real 0m1.222s user 0m0.006s sys 0m0.187s 'ls -l' 100 files with statahead: 1 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 30 fname_total: 109 hit_total: 4477 miss_total: 1287 swrong=2 ewrong=2 - open/close 643 (time 1788218231.13 total 10.01 last 64.27) total: 900 open/close in 15.00 seconds: 59.99 ops/second llite.lustre-ffff8def10755000.statahead_max=0 1001 real 0m24.648s user 0m0.040s sys 0m2.947s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff8def10755000.statahead_max=32 llite.lustre-ffff8def10755000.statahead_max=32 1001 real 0m13.101s user 0m0.044s sys 0m1.597s 'ls -l' 1000 files with statahead: 13 sec statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 31 fname_total: 109 hit_total: 5474 miss_total: 1290 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff8def10755000.statahead_stats= statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 31 fname_total: 109 hit_total: 5474 miss_total: 1290 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats= snapshot_time: 1788218316.262155856 secs.nsecs start_time: 1788218206.640578390 secs.nsecs elapsed_time: 109.621577466 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 578 (time 1788218329.04 total 10.01 last 57.73) - open/close 1076 (time 1788218339.05 total 20.02 last 49.76) - open/close 1706 (time 1788218349.07 total 30.04 last 62.89) - open/close 2276 (time 1788218359.07 total 40.04 last 56.97) - open/close 2896 (time 1788218369.09 total 50.06 last 61.90) - open/close 3491 (time 1788218379.09 total 60.06 last 59.48) - open/close 4141 (time 1788218389.09 total 70.06 last 64.99) - open/close 4742 (time 1788218399.10 total 80.07 last 60.08) - open/close 5357 (time 1788218409.10 total 90.07 last 61.49) - open/close 5973 (time 1788218419.12 total 100.09 last 61.50) - open/close 6623 (time 1788218429.12 total 110.09 last 64.99) - open/close 7219 (time 1788218439.13 total 120.10 last 59.55) - open/close 7812 (time 1788218449.13 total 130.10 last 59.30) - open/close 8440 (time 1788218459.13 total 140.10 last 62.77) - open/close 9083 (time 1788218469.13 total 150.10 last 64.29) - open/close 9667 (time 1788218479.14 total 160.11 last 58.36) total: 10000 open/close in 165.47 seconds: 60.43 ops/second llite.lustre-ffff8def10755000.statahead_batch_max=0 llite.lustre-ffff8def10755000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.703s user 0m0.343s sys 0m14.843s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8def10755000.statahead_batch_max=32 llite.lustre-ffff8def10755000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.305s user 0m0.316s sys 0m13.869s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats= snapshot_time: 1788218686.150281308 secs.nsecs start_time: 1788218563.214368060 secs.nsecs elapsed_time: 122.935913248 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=0 llite.lustre-ffff8def10755000.statahead_max=2048 llite.lustre-ffff8def10755000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=0 total: 100 open/close in 3.52 seconds: 28.38 ops/second llite.lustre-ffff8def10755000.statahead_max=0 101 real 0m3.187s user 0m0.006s sys 0m0.332s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8def10755000.statahead_max=2048 llite.lustre-ffff8def10755000.statahead_max=2048 101 real 0m1.262s user 0m0.001s sys 0m0.168s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 493 (time 1788219071.23 total 10.00 last 49.28) total: 900 open/close in 18.54 seconds: 48.53 ops/second llite.lustre-ffff8def10755000.statahead_max=0 1001 real 0m27.067s user 0m0.038s sys 0m3.064s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8def10755000.statahead_max=2048 llite.lustre-ffff8def10755000.statahead_max=2048 1001 real 0m7.179s user 0m0.023s sys 0m1.341s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff8def10755000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats= snapshot_time: 1788219155.539996563 secs.nsecs start_time: 1788218689.044196948 secs.nsecs elapsed_time: 466.495799615 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 2 18 45 64: 1 9 54 128: 1 9 63 256: 4 36 100 - open/close 456 (time 1788219169.68 total 10.02 last 45.50) - open/close 997 (time 1788219179.70 total 20.04 last 53.99) - open/close 1607 (time 1788219189.70 total 30.04 last 60.99) - open/close 2223 (time 1788219199.70 total 40.04 last 61.60) - open/close 2770 (time 1788219209.71 total 50.05 last 54.68) - open/close 3164 (time 1788219219.76 total 60.11 last 39.18) - open/close 3654 (time 1788219229.79 total 70.13 last 48.87) - open/close 4236 (time 1788219239.79 total 80.14 last 58.18) - open/close 4816 (time 1788219249.81 total 90.15 last 57.92) - open/close 5368 (time 1788219259.81 total 100.15 last 55.19) - open/close 5918 (time 1788219269.84 total 110.18 last 54.85) - open/close 6506 (time 1788219279.85 total 120.19 last 58.74) - open/close 7018 (time 1788219289.86 total 130.20 last 51.15) - open/close 7601 (time 1788219299.87 total 140.21 last 58.23) - open/close 8145 (time 1788219309.90 total 150.24 last 54.24) - open/close 8672 (time 1788219319.90 total 160.24 last 52.70) - open/close 9212 (time 1788219329.91 total 170.25 last 53.94) - open/close 9723 (time 1788219339.91 total 180.25 last 51.09) total: 10000 open/close in 187.08 seconds: 53.45 ops/second llite.lustre-ffff8def10755000.statahead_batch_max=0 llite.lustre-ffff8def10755000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.871s user 0m0.339s sys 0m14.362s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8def10755000.statahead_batch_max=256 llite.lustre-ffff8def10755000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8def10755000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.376s user 0m0.354s sys 0m14.448s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8def10755000.batch_stats= snapshot_time: 1788219491.778100234 secs.nsecs start_time: 1788219427.207033999 secs.nsecs elapsed_time: 64.571066235 secs.nsecs subreqs per batch batches %% cum %% 1: 129 49 49 2: 35 13 63 4: 17 6 69 8: 11 4 74 16: 14 5 79 32: 16 6 85 64: 1 0 86 128: 0 0 86 256: 36 13 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8def10755000.statahead_batch_max=64 llite.lustre-ffff8def10755000.statahead_max=128