== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:08:52 (1788934132) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=0 llite.lustre-ffff8d0e11c0f800.statahead_max=32 llite.lustre-ffff8d0e11c0f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=0 total: 100 open/close in 2.93 seconds: 34.12 ops/second llite.lustre-ffff8d0e11c0f800.statahead_max=0 101 real 0m2.118s user 0m0.003s sys 0m0.312s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8d0e11c0f800.statahead_max=32 llite.lustre-ffff8d0e11c0f800.statahead_max=32 101 real 0m1.356s user 0m0.005s sys 0m0.174s 'ls -l' 100 files with statahead: 2 sec statahead total: 125 statahead wrong: 2 agl total: 119 list_total: 30 fname_total: 63 hit_total: 4205 miss_total: 770 swrong=2 ewrong=2 - open/close 539 (time 1788934160.98 total 10.00 last 53.89) total: 900 open/close in 14.99 seconds: 60.02 ops/second llite.lustre-ffff8d0e11c0f800.statahead_max=0 1001 real 0m25.186s user 0m0.036s sys 0m2.912s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff8d0e11c0f800.statahead_max=32 llite.lustre-ffff8d0e11c0f800.statahead_max=32 1001 real 0m9.721s user 0m0.041s sys 0m1.323s 'ls -l' 1000 files with statahead: 10 sec statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 63 hit_total: 5204 miss_total: 771 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8d0e11c0f800.statahead_stats= statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 31 fname_total: 63 hit_total: 5204 miss_total: 771 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats= snapshot_time: 1788934237.587845823 secs.nsecs start_time: 1788934137.122047030 secs.nsecs elapsed_time: 100.465798793 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 665 (time 1788934249.96 total 10.01 last 66.46) - open/close 1383 (time 1788934259.96 total 20.01 last 71.79) - open/close 2128 (time 1788934269.97 total 30.02 last 74.42) - open/close 2908 (time 1788934279.97 total 40.02 last 77.98) - open/close 3709 (time 1788934289.97 total 50.02 last 80.09) - open/close 4454 (time 1788934299.98 total 60.03 last 74.47) - open/close 5143 (time 1788934309.98 total 70.03 last 68.87) - open/close 5696 (time 1788934319.99 total 80.04 last 55.28) - open/close 6342 (time 1788934329.99 total 90.04 last 64.57) - open/close 7001 (time 1788934340.00 total 100.05 last 65.85) - open/close 7602 (time 1788934350.00 total 110.05 last 60.09) - open/close 8303 (time 1788934360.03 total 120.08 last 69.88) - open/close 8911 (time 1788934370.06 total 130.11 last 60.61) - open/close 9507 (time 1788934380.07 total 140.12 last 59.56) total: 10000 open/close in 147.44 seconds: 67.83 ops/second llite.lustre-ffff8d0e11c0f800.statahead_batch_max=0 llite.lustre-ffff8d0e11c0f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.000s user 0m0.305s sys 0m14.412s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d0e11c0f800.statahead_batch_max=32 llite.lustre-ffff8d0e11c0f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m48.785s user 0m0.294s sys 0m13.334s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats= snapshot_time: 1788934573.915524334 secs.nsecs start_time: 1788934454.906765080 secs.nsecs elapsed_time: 119.008759254 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=0 llite.lustre-ffff8d0e11c0f800.statahead_max=2048 llite.lustre-ffff8d0e11c0f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=0 total: 100 open/close in 3.51 seconds: 28.50 ops/second llite.lustre-ffff8d0e11c0f800.statahead_max=0 101 real 0m2.753s user 0m0.010s sys 0m0.318s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8d0e11c0f800.statahead_max=2048 llite.lustre-ffff8d0e11c0f800.statahead_max=2048 101 real 0m1.296s user 0m0.005s sys 0m0.153s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 668 (time 1788934895.17 total 10.00 last 66.80) total: 900 open/close in 13.46 seconds: 66.85 ops/second llite.lustre-ffff8d0e11c0f800.statahead_max=0 1001 real 0m25.202s user 0m0.034s sys 0m3.077s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff8d0e11c0f800.statahead_max=2048 llite.lustre-ffff8d0e11c0f800.statahead_max=2048 1001 real 0m7.381s user 0m0.032s sys 0m1.531s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8d0e11c0f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats= snapshot_time: 1788934972.152164440 secs.nsecs start_time: 1788934576.723860721 secs.nsecs elapsed_time: 395.428303719 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 0 0 33 64: 1 11 44 128: 1 11 55 256: 4 44 100 - open/close 661 (time 1788934985.20 total 10.01 last 66.01) - open/close 1175 (time 1788934995.22 total 20.03 last 51.33) - open/close 1846 (time 1788935005.22 total 30.03 last 67.07) - open/close 2564 (time 1788935015.24 total 40.05 last 71.67) - open/close 3335 (time 1788935025.26 total 50.07 last 76.96) - open/close 3942 (time 1788935035.26 total 60.07 last 60.69) - open/close 4646 (time 1788935045.26 total 70.08 last 70.37) - open/close 5342 (time 1788935055.27 total 80.08 last 69.58) - open/close 5946 (time 1788935065.28 total 90.10 last 60.29) - open/close 6614 (time 1788935075.33 total 100.14 last 66.49) - open/close 7282 (time 1788935085.34 total 110.15 last 66.77) - open/close 7992 (time 1788935095.34 total 120.15 last 70.99) - open/close 8719 (time 1788935105.35 total 130.16 last 72.61) - open/close 9343 (time 1788935115.36 total 140.17 last 62.35) - open/close 9927 (time 1788935125.36 total 150.18 last 58.36) total: 10000 open/close in 151.82 seconds: 65.87 ops/second llite.lustre-ffff8d0e11c0f800.statahead_batch_max=0 llite.lustre-ffff8d0e11c0f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.198s user 0m0.312s sys 0m13.876s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d0e11c0f800.statahead_batch_max=256 llite.lustre-ffff8d0e11c0f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.796s user 0m0.305s sys 0m13.742s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8d0e11c0f800.batch_stats= snapshot_time: 1788935270.047442704 secs.nsecs start_time: 1788935198.287394276 secs.nsecs elapsed_time: 71.760048428 secs.nsecs subreqs per batch batches %% cum %% 1: 131 47 47 2: 28 10 57 4: 33 12 69 8: 14 5 74 16: 8 2 77 32: 20 7 85 64: 6 2 87 128: 1 0 87 256: 34 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8d0e11c0f800.statahead_batch_max=64 llite.lustre-ffff8d0e11c0f800.statahead_max=128