== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:57:01 (1787399821) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=0 llite.lustre-ffff8f3220480000.statahead_max=32 llite.lustre-ffff8f3220480000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.82 seconds: 35.44 ops/second llite.lustre-ffff8f3220480000.statahead_max=0 101 real 0m2.262s user 0m0.003s sys 0m0.323s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8f3220480000.statahead_max=32 llite.lustre-ffff8f3220480000.statahead_max=32 101 real 0m1.331s user 0m0.003s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 107 statahead wrong: 2 agl total: 101 list_total: 31 fname_total: 73 hit_total: 4424 miss_total: 1272 swrong=2 ewrong=2 - open/close 849 (time 1787399849.12 total 10.00 last 84.89) total: 900 open/close in 10.94 seconds: 82.24 ops/second llite.lustre-ffff8f3220480000.statahead_max=0 1001 real 0m20.700s user 0m0.031s sys 0m3.125s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8f3220480000.statahead_max=32 llite.lustre-ffff8f3220480000.statahead_max=32 1001 real 0m9.922s user 0m0.029s sys 0m1.570s 'ls -l' 1000 files with statahead: 10 sec statahead total: 108 statahead wrong: 2 agl total: 102 list_total: 32 fname_total: 73 hit_total: 5422 miss_total: 1274 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff8f3220480000.statahead_stats= statahead total: 108 statahead wrong: 2 agl total: 102 list_total: 32 fname_total: 73 hit_total: 5422 miss_total: 1274 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787399911.629789930 secs.nsecs start_time: 1787399825.473398743 secs.nsecs elapsed_time: 86.156391187 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 3 7 15 16: 19 48 64 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787399911.629846869 secs.nsecs start_time: 1787399825.473547502 secs.nsecs elapsed_time: 86.156299367 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 1 2 7 4: 1 2 10 8: 3 7 17 16: 20 50 67 32: 13 32 100 - open/close 856 (time 1787399923.96 total 10.00 last 85.58) - open/close 1730 (time 1787399933.97 total 20.01 last 87.29) - open/close 2630 (time 1787399943.97 total 30.02 last 89.98) - open/close 3569 (time 1787399954.00 total 40.04 last 93.69) - open/close 4453 (time 1787399964.01 total 50.05 last 88.28) - open/close 5384 (time 1787399974.01 total 60.06 last 93.07) - open/close 6137 (time 1787399984.02 total 70.06 last 75.27) - open/close 7021 (time 1787399994.03 total 80.07 last 88.29) - open/close 7903 (time 1787400004.03 total 90.07 last 88.18) - open/close 8746 (time 1787400014.03 total 100.08 last 84.27) - open/close 9698 (time 1787400024.04 total 110.08 last 95.14) total: 10000 open/close in 113.34 seconds: 88.23 ops/second llite.lustre-ffff8f3220480000.statahead_batch_max=0 llite.lustre-ffff8f3220480000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.633s user 0m0.289s sys 0m16.773s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8f3220480000.statahead_batch_max=32 llite.lustre-ffff8f3220480000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.290s user 0m0.306s sys 0m15.477s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400224.079857018 secs.nsecs start_time: 1787400097.662097334 secs.nsecs elapsed_time: 126.417759684 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400224.080513251 secs.nsecs start_time: 1787400097.662222817 secs.nsecs elapsed_time: 126.418290434 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=0 llite.lustre-ffff8f3220480000.statahead_max=2048 llite.lustre-ffff8f3220480000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.72 seconds: 36.70 ops/second llite.lustre-ffff8f3220480000.statahead_max=0 101 real 0m2.488s user 0m0.004s sys 0m0.345s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8f3220480000.statahead_max=2048 llite.lustre-ffff8f3220480000.statahead_max=2048 101 real 0m0.905s user 0m0.005s sys 0m0.187s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 - open/close 840 (time 1787400516.45 total 10.01 last 83.93) total: 900 open/close in 10.89 seconds: 82.62 ops/second llite.lustre-ffff8f3220480000.statahead_max=0 1001 real 0m23.616s user 0m0.032s sys 0m3.003s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8f3220480000.statahead_max=2048 llite.lustre-ffff8f3220480000.statahead_max=2048 1001 real 0m7.881s user 0m0.031s sys 0m1.716s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8f3220480000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400580.982252546 secs.nsecs start_time: 1787400227.128138805 secs.nsecs elapsed_time: 353.854113741 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 33 33 8: 1 11 44 16: 2 22 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400580.982312091 secs.nsecs start_time: 1787400227.128292466 secs.nsecs elapsed_time: 353.854019625 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 11 11 4: 1 11 22 8: 3 33 55 16: 1 11 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 - open/close 721 (time 1787400593.03 total 10.01 last 72.01) - open/close 1531 (time 1787400603.04 total 20.02 last 80.95) - open/close 2309 (time 1787400613.04 total 30.02 last 77.75) - open/close 3113 (time 1787400623.05 total 40.03 last 80.32) - open/close 4000 (time 1787400633.06 total 50.04 last 88.65) - open/close 4866 (time 1787400643.06 total 60.04 last 86.58) - open/close 5839 (time 1787400653.07 total 70.05 last 97.24) - open/close 6809 (time 1787400663.07 total 80.05 last 96.97) - open/close 7764 (time 1787400673.07 total 90.05 last 95.49) - open/close 8616 (time 1787400683.07 total 100.05 last 85.20) - open/close 9451 (time 1787400693.08 total 110.06 last 83.43) total: 10000 open/close in 116.75 seconds: 85.65 ops/second llite.lustre-ffff8f3220480000.statahead_batch_max=0 llite.lustre-ffff8f3220480000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.558s user 0m0.287s sys 0m17.418s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8f3220480000.statahead_batch_max=256 llite.lustre-ffff8f3220480000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8f3220480000.stats=clear mdc.lustre-MDT0001-mdc-ffff8f3220480000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.196s user 0m0.301s sys 0m16.498s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400844.258053252 secs.nsecs start_time: 1787400768.884521243 secs.nsecs elapsed_time: 75.373532009 secs.nsecs subreqs per batch batches %% cum %% 1: 295 55 55 2: 73 13 69 4: 69 12 81 8: 29 5 87 16: 11 2 89 32: 19 3 93 64: 4 0 93 128: 0 0 93 256: 33 6 100 mdc.lustre-MDT0001-mdc-ffff8f3220480000.batch_stats= snapshot_time: 1787400844.258118802 secs.nsecs start_time: 1787400768.885819858 secs.nsecs elapsed_time: 75.372298944 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8f3220480000.statahead_batch_max=64 llite.lustre-ffff8f3220480000.statahead_max=128