-----============= acceptance-small: lustre-rsync-test ============----- Fri Sep 11 08:21:55 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg327-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex': No such file or directory excepting tests: 2b === lustre-rsync-test: start setup 08:22:12 (1789129332) === oleg327-client.virtnet: executing check_config_client /mnt/lustre oleg327-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg327-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff984e0a0d8000.idle_timeout=debug osc.lustre-OST0001-osc-ffff984e0a0d8000.idle_timeout=debug disable quota as required oleg327-server: oleg327-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 Setting lod.lustre-MDT0000-mdtlov.enable_compr_rotational from 0 to 1 Waiting 90s for '1' Updated after 2s: want '1' got '1' Setting lod.lustre-MDT0001-mdtlov.enable_compr_rotational from 0 to 1 Waiting 90s for '1' Updated after 2s: want '1' got '1' === lustre-rsync-test: finish setup 08:23:04 (1789129384) === nobody:x:65534: == lustre-rsync-test test 1A: Simple Replication ========= 08:23:06 (1789129386) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000: Registered changelog user cl1 Replication #1 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 26 Replication #2 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 26 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0001: changelog user 'cl1' not found lustre-MDT0000: changelog user 'cl1' not found PASS 1A (22s) == lustre-rsync-test test 1a: Replicate create/delete operations in ROOT directory ========================================================== 08:23:28 (1789129408) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' lustre-MDT0000: Registered changelog user cl2 Replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 11 File: /mnt/target/root-dir Size: 0 Blocks: 0 IO Block: 4096 directory Device: 2bh/43d Inode: 49493 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-11 08:23:35.000000000 -0400 Modify: 2026-09-11 08:23:35.000000000 -0400 Change: 2026-09-11 08:23:35.628000000 -0400 Birth: - File: /mnt/target/root-file Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 2bh/43d Inode: 48731 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-11 08:23:35.000000000 -0400 Modify: 2026-09-11 08:23:35.000000000 -0400 Change: 2026-09-11 08:23:35.676000000 -0400 Birth: - stat: cannot statx '/mnt/target/root-file2': No such file or directory File: /mnt/target/root-file3 Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 2bh/43d Inode: 47997 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-11 08:23:35.000000000 -0400 Modify: 2026-09-11 08:23:35.000000000 -0400 Change: 2026-09-11 08:23:35.718000000 -0400 Birth: - stat: cannot statx '/mnt/target/root-dir1': No such file or directory stat: cannot statx '/mnt/target/root-file4': No such file or directory lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0001: changelog user 'cl2' not found lustre-MDT0000: changelog user 'cl2' not found PASS 1a (20s) == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 08:23:48 (1789129428) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' lustre-MDT0000: Registered changelog user cl3 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri Sep 11 08:23:59 EDT 2026 waiting for dbench pid 12772 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 51 4.19 MB/sec warmup 1 sec latency 75.155 ms 2 156 6.54 MB/sec warmup 2 sec latency 89.198 ms 2 207 5.40 MB/sec warmup 3 sec latency 149.957 ms 2 289 5.35 MB/sec warmup 4 sec latency 94.082 ms 2 349 5.01 MB/sec warmup 5 sec latency 141.816 ms 2 413 4.86 MB/sec warmup 6 sec latency 136.080 ms 2 473 4.72 MB/sec warmup 7 sec latency 133.476 ms 2 519 4.46 MB/sec warmup 8 sec latency 156.762 ms 2 648 4.67 MB/sec warmup 9 sec latency 114.715 ms 2 702 4.23 MB/sec warmup 10 sec latency 110.952 ms 2 740 3.87 MB/sec warmup 11 sec latency 148.085 ms 2 857 0.20 MB/sec execute 1 sec latency 143.914 ms 2 892 0.21 MB/sec execute 2 sec latency 134.339 ms 2 928 0.20 MB/sec execute 3 sec latency 121.814 ms 2 985 0.22 MB/sec execute 4 sec latency 105.837 ms 2 1064 0.57 MB/sec execute 5 sec latency 102.904 ms 2 1136 0.50 MB/sec execute 6 sec latency 80.091 ms 2 1188 0.45 MB/sec execute 7 sec latency 99.386 ms 2 1240 0.40 MB/sec execute 8 sec latency 81.517 ms 2 1393 1.01 MB/sec execute 9 sec latency 113.812 ms 2 1456 0.91 MB/sec execute 10 sec latency 105.639 ms 2 1510 0.83 MB/sec execute 11 sec latency 112.110 ms 2 1612 0.78 MB/sec execute 12 sec latency 63.424 ms 2 1687 0.73 MB/sec execute 13 sec latency 101.117 ms 2 1769 0.69 MB/sec execute 14 sec latency 97.198 ms 2 1850 0.65 MB/sec execute 15 sec latency 100.047 ms 2 1944 0.62 MB/sec execute 16 sec latency 80.376 ms 2 2005 0.59 MB/sec execute 17 sec latency 79.610 ms 2 2097 0.57 MB/sec execute 18 sec latency 115.306 ms 2 2161 0.55 MB/sec execute 19 sec latency 95.810 ms 2 2274 0.58 MB/sec execute 20 sec latency 66.025 ms 2 2365 0.61 MB/sec execute 21 sec latency 98.116 ms 2 2421 0.60 MB/sec execute 22 sec latency 90.521 ms 2 2480 0.63 MB/sec execute 23 sec latency 105.831 ms 2 2579 0.62 MB/sec execute 24 sec latency 82.615 ms 2 2761 0.84 MB/sec execute 25 sec latency 169.274 ms 2 2969 0.92 MB/sec execute 26 sec latency 75.688 ms 2 3180 0.93 MB/sec execute 27 sec latency 93.175 ms 2 3306 0.99 MB/sec execute 28 sec latency 182.449 ms 2 3441 1.06 MB/sec execute 29 sec latency 70.861 ms 2 3575 1.13 MB/sec execute 30 sec latency 195.528 ms 2 3688 1.11 MB/sec execute 31 sec latency 58.857 ms 2 3776 1.14 MB/sec execute 32 sec latency 87.878 ms 2 3836 1.11 MB/sec execute 33 sec latency 84.751 ms 2 3918 1.09 MB/sec execute 34 sec latency 90.979 ms 2 3974 1.06 MB/sec execute 35 sec latency 136.954 ms 2 4041 1.03 MB/sec execute 36 sec latency 77.228 ms 2 4142 1.00 MB/sec execute 37 sec latency 68.645 ms 2 4239 0.99 MB/sec execute 38 sec latency 133.806 ms 2 4297 0.97 MB/sec execute 39 sec latency 98.182 ms 2 4365 0.95 MB/sec execute 40 sec latency 85.639 ms 2 4455 0.94 MB/sec execute 41 sec latency 63.725 ms 2 4529 0.92 MB/sec execute 42 sec latency 86.010 ms 2 4624 0.95 MB/sec execute 43 sec latency 80.753 ms 2 4722 0.94 MB/sec execute 44 sec latency 86.797 ms 2 4805 0.92 MB/sec execute 45 sec latency 53.759 ms 2 4967 1.03 MB/sec execute 46 sec latency 89.626 ms 2 5025 1.01 MB/sec execute 47 sec latency 97.656 ms 2 5125 0.99 MB/sec execute 48 sec latency 77.495 ms 2 5183 0.97 MB/sec execute 49 sec latency 82.213 ms 2 5261 0.95 MB/sec execute 50 sec latency 74.642 ms 2 5335 0.94 MB/sec execute 51 sec latency 62.835 ms 2 5387 0.92 MB/sec execute 52 sec latency 97.336 ms 2 5454 0.91 MB/sec execute 53 sec latency 90.599 ms 2 5559 0.89 MB/sec execute 54 sec latency 103.264 ms 2 5624 0.88 MB/sec execute 55 sec latency 109.948 ms 2 5718 0.87 MB/sec execute 56 sec latency 92.694 ms 2 5822 0.86 MB/sec execute 57 sec latency 72.624 ms 2 5925 0.88 MB/sec execute 58 sec latency 71.309 ms 2 5994 0.87 MB/sec execute 59 sec latency 67.498 ms 2 cleanup 60 sec 2 cleanup 61 sec 0 cleanup 62 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 1852 33.973 195.339 Close 1436 8.899 65.747 Rename 78 54.499 121.789 Unlink 315 12.881 41.818 Qpathinfo 1786 8.775 120.412 Qfileinfo 308 0.683 7.609 Qfsinfo 263 0.680 34.997 Sfileinfo 181 38.422 128.856 Find 637 2.156 81.812 WriteX 942 6.613 120.368 ReadX 2654 0.051 3.513 LockX 4 4.788 5.316 UnlockX 4 5.016 7.060 Flush 122 40.917 112.078 Throughput 0.871394 MB/sec 2 clients 2 procs max_latency=195.528 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri Sep 11 08:25:14 EDT 2026 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory 'clients/client1/~dmtmp/WORDPRO' removed directory 'clients/client1/~dmtmp/COREL' removed directory 'clients/client1/~dmtmp/PARADOX' removed directory 'clients/client1/~dmtmp/WORD' removed directory 'clients/client1/~dmtmp/SEED' removed directory 'clients/client1/~dmtmp/ACCESS' removed directory 'clients/client1/~dmtmp/PM' removed directory 'clients/client1/~dmtmp/EXCEL' removed directory 'clients/client1/~dmtmp/PWRPNT' removed directory 'clients/client1/~dmtmp' removed directory 'clients/client1' removed directory 'clients/client0/~dmtmp/WORDPRO' removed directory 'clients/client0/~dmtmp/COREL' removed directory 'clients/client0/~dmtmp/PARADOX' removed directory 'clients/client0/~dmtmp/WORD' removed directory 'clients/client0/~dmtmp/SEED' removed directory 'clients/client0/~dmtmp/ACCESS' removed directory 'clients/client0/~dmtmp/PM' removed directory 'clients/client0/~dmtmp/EXCEL' removed directory 'clients/client0/~dmtmp/PWRPNT' removed directory 'clients/client0/~dmtmp' removed directory 'clients/client0' removed directory 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl3 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 6 seconds Changelog records consumed: 2648 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0001: changelog user 'cl3' not found lustre-MDT0000: changelog user 'cl3' not found PASS 2a (106s) SKIP: lustre-rsync-test test_2b skipping ALWAYS excluded test 2b == lustre-rsync-test test 2c: Replicate files while dbench is running. ========================================================== 08:25:36 (1789129536) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' lustre-MDT0000: Registered changelog user cl4 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2c.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2c.lustre-rsync-test at Fri Sep 11 08:25:45 EDT 2026 waiting for dbench pid 13797 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 109 10.21 MB/sec warmup 1 sec latency 74.750 ms 2 169 6.89 MB/sec warmup 2 sec latency 145.179 ms 2 224 5.73 MB/sec warmup 3 sec latency 339.084 ms 2 300 5.53 MB/sec warmup 4 sec latency 102.088 ms 2 369 5.26 MB/sec warmup 5 sec latency 90.722 ms 2 454 5.29 MB/sec warmup 6 sec latency 82.760 ms 2 539 5.41 MB/sec warmup 7 sec latency 107.954 ms Running lustre_rsync 2 677 5.29 MB/sec warmup 8 sec latency 77.929 ms 2 752 4.73 MB/sec warmup 9 sec latency 72.831 ms 2 849 4.31 MB/sec warmup 10 sec latency 51.956 ms 2 913 3.95 MB/sec warmup 11 sec latency 67.495 ms 2 1121 2.14 MB/sec execute 1 sec latency 61.881 ms 2 1191 1.13 MB/sec execute 2 sec latency 57.319 ms 2 1248 0.79 MB/sec execute 3 sec latency 78.473 ms 2 1403 2.05 MB/sec execute 4 sec latency 135.140 ms 2 1455 1.65 MB/sec execute 5 sec latency 153.104 ms 2 1528 1.40 MB/sec execute 6 sec latency 79.470 ms 2 1618 1.22 MB/sec execute 7 sec latency 76.494 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 12 seconds Changelog records consumed: 271 2 1683 1.08 MB/sec execute 8 sec latency 83.213 ms 2 1731 0.97 MB/sec execute 9 sec latency 75.600 ms 2 1793 0.88 MB/sec execute 10 sec latency 106.329 ms 2 1853 0.81 MB/sec execute 11 sec latency 121.983 ms 2 1965 0.76 MB/sec execute 12 sec latency 68.808 ms 13797 13798 13799 Running lustre_rsync 2 2018 0.71 MB/sec execute 13 sec latency 77.494 ms 2 2112 0.68 MB/sec execute 14 sec latency 96.144 ms 2 2190 0.65 MB/sec execute 15 sec latency 71.808 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 2959 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 572 2 2338 0.74 MB/sec execute 16 sec latency 81.392 ms 2 2385 0.71 MB/sec execute 17 sec latency 96.028 ms 2 2447 0.69 MB/sec execute 18 sec latency 100.598 ms 2 2572 0.74 MB/sec execute 19 sec latency 67.722 ms 2 2772 1.00 MB/sec execute 20 sec latency 70.852 ms 13797 13798 13799 Running lustre_rsync 2 2950 1.10 MB/sec execute 21 sec latency 99.113 ms 2 3012 1.05 MB/sec execute 22 sec latency 142.980 ms 2 3223 1.08 MB/sec execute 23 sec latency 88.722 ms 2 3347 1.12 MB/sec execute 24 sec latency 95.794 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 3530 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 301 2 3508 1.31 MB/sec execute 25 sec latency 87.895 ms 2 3632 1.28 MB/sec execute 26 sec latency 54.572 ms 2 3692 1.24 MB/sec execute 27 sec latency 151.092 ms 2 3778 1.27 MB/sec execute 28 sec latency 84.378 ms 2 3852 1.23 MB/sec execute 29 sec latency 70.442 ms 13797 13798 13799 Running lustre_rsync 2 3931 1.21 MB/sec execute 30 sec latency 77.215 ms 2 3995 1.17 MB/sec execute 31 sec latency 108.511 ms 2 4059 1.13 MB/sec execute 32 sec latency 91.605 ms 2 4123 1.10 MB/sec execute 33 sec latency 121.693 ms 2 4212 1.07 MB/sec execute 34 sec latency 73.845 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 3830 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 383 2 4276 1.05 MB/sec execute 35 sec latency 132.239 ms 2 4336 1.03 MB/sec execute 36 sec latency 101.355 ms 2 4388 1.01 MB/sec execute 37 sec latency 87.132 ms 2 4457 0.99 MB/sec execute 38 sec latency 91.984 ms 2 4526 0.97 MB/sec execute 39 sec latency 83.279 ms 13797 13798 13799 Running lustre_rsync 2 4564 0.95 MB/sec execute 40 sec latency 96.096 ms 2 4650 0.98 MB/sec execute 41 sec latency 99.808 ms 2 4711 0.96 MB/sec execute 42 sec latency 77.138 ms 2 4763 0.94 MB/sec execute 43 sec latency 96.175 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 4212 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 337 2 4824 0.92 MB/sec execute 44 sec latency 67.002 ms 2 4976 1.03 MB/sec execute 45 sec latency 73.472 ms 2 5036 1.01 MB/sec execute 46 sec latency 116.289 ms 2 5128 0.99 MB/sec execute 47 sec latency 72.361 ms 2 5210 0.97 MB/sec execute 48 sec latency 81.028 ms 13797 13798 13799 Running lustre_rsync 2 5281 0.96 MB/sec execute 49 sec latency 77.738 ms 2 5340 0.94 MB/sec execute 50 sec latency 97.456 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 4548 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 258 2 5405 0.92 MB/sec execute 51 sec latency 82.984 ms 2 5544 0.91 MB/sec execute 52 sec latency 60.049 ms 2 5586 0.90 MB/sec execute 53 sec latency 107.168 ms 2 5685 0.88 MB/sec execute 54 sec latency 68.066 ms 2 5740 0.87 MB/sec execute 55 sec latency 99.296 ms 13797 13798 13799 Running lustre_rsync 2 5840 0.87 MB/sec execute 56 sec latency 71.829 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 4805 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 232 2 5923 0.88 MB/sec execute 57 sec latency 118.554 ms 2 5986 0.87 MB/sec execute 58 sec latency 74.865 ms 2 6043 0.88 MB/sec execute 59 sec latency 83.893 ms 2 cleanup 60 sec 2 cleanup 61 sec 13797 13798 13799 Running lustre_rsync 2 cleanup 62 sec 0 cleanup 63 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 1794 35.914 153.087 Close 1385 8.617 38.952 Rename 76 54.418 101.157 Unlink 307 14.352 44.561 Qpathinfo 1773 9.228 151.061 Qfileinfo 305 0.622 9.513 Qfsinfo 257 0.798 20.269 Sfileinfo 165 38.486 91.961 Find 619 2.297 121.664 WriteX 910 6.256 73.852 ReadX 2620 0.051 3.934 LockX 4 5.881 9.265 UnlockX 4 4.963 10.580 Flush 107 43.039 132.216 Throughput 0.876217 MB/sec 2 clients 2 procs max_latency=153.104 ms stopping dbench in /mnt/lustre/d2c.lustre-rsync-test at Fri Sep 11 08:27:00 EDT 2026 with return code 0 clean dbench files in /mnt/lustre/d2c.lustre-rsync-test /mnt/lustre/d2c.lustre-rsync-test /mnt/lustre/d2c.lustre-rsync-test removed directory 'clients/client1/~dmtmp/WORDPRO' removed directory 'clients/client1/~dmtmp/COREL' removed directory 'clients/client1/~dmtmp/PARADOX' removed directory 'clients/client1/~dmtmp/WORD' removed directory 'clients/client1/~dmtmp/SEED' removed directory 'clients/client1/~dmtmp/ACCESS' removed directory 'clients/client1/~dmtmp/PM' removed directory 'clients/client1/~dmtmp/EXCEL' removed directory 'clients/client1/~dmtmp/PWRPNT' removed directory 'clients/client1/~dmtmp' removed directory 'clients/client1' removed directory 'clients/client0/~dmtmp/WORDPRO' removed directory 'clients/client0/~dmtmp/COREL' removed directory 'clients/client0/~dmtmp/PARADOX' removed directory 'clients/client0/~dmtmp/WORD' removed directory 'clients/client0/~dmtmp/SEED' removed directory 'clients/client0/~dmtmp/ACCESS' removed directory 'clients/client0/~dmtmp/PM' removed directory 'clients/client0/~dmtmp/EXCEL' removed directory 'clients/client0/~dmtmp/PWRPNT' removed directory 'clients/client0/~dmtmp' removed directory 'clients/client0' removed directory 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2c.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 5036 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 246 Running lustre_rsync Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 5281 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 78 Running final lustre_rsync Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 5358 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 1 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0001: changelog user 'cl4' not found lustre-MDT0000: changelog user 'cl4' not found PASS 2c (110s) == lustre-rsync-test test 3a: Replicate files created by createmany ========================================================== 08:27:27 (1789129647) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' lustre-MDT0000: Registered changelog user cl5 - open/close 710 (time 1789129664.97 total 10.00 last 70.99) total: 1000 open/close in 13.97 seconds: 71.56 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl5 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 71 seconds Changelog records consumed: 2000 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0001: changelog user 'cl5' not found lustre-MDT0000: changelog user 'cl5' not found PASS 3a (133s) == lustre-rsync-test test 3b: Replicate files created by writemany ========================================================== 08:29:39 (1789129779) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' lustre-MDT0000: Registered changelog user cl6 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl6 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 43 seconds Changelog records consumed: 7785 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0001: changelog user 'cl6' not found lustre-MDT0000: changelog user 'cl6' not found PASS 3b (124s) == lustre-rsync-test test 3c: Replicate files created by createmany/unlinkmany ========================================================== 08:31:43 (1789129903) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' lustre-MDT0000: Registered changelog user cl7 - open/close 715 (time 1789129923.38 total 10.00 last 71.48) total: 1000 open/close in 13.29 seconds: 75.22 ops/second - unlinked 0 (time 1789129932 ; total 0 ; last 0) total: 1000 unlinks in 9 seconds: 111.111115 unlinks/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl7 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 17 seconds Changelog records consumed: 3000 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0001: changelog user 'cl7' not found lustre-MDT0000: changelog user 'cl7' not found PASS 3c (72s) == lustre-rsync-test test 4: Replicate files created by iozone ========================================================== 08:32:55 (1789129975) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lustre-MDT0000: Registered changelog user cl8 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none /mnt/lustre/d4.lustre-rsync-test/d0.iozone-oleg327-client.virtnet stripe_count: 1 stripe_size: 4194304 pattern: stripe_offset: -1 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl8 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 477 /home/green/git/lustre-release/lustre/tests/lustre-rsync-test.sh: line 473: 17694 Killed END_RUN_FILE=${DIR}/$tdir/run LOAD_PID_FILE=${DIR}/$tdir/pid MOUNT=${DIR}/$tdir RECOVERY_SCALE_ENABLE_REMOTE_DIRS=false RECOVERY_SCALE_ENABLE_STRIPED_DIRS=false run_iozone.sh Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl8 Starting changelog record: 18620 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 190 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0001: changelog user 'cl8' not found lustre-MDT0000: changelog user 'cl8' not found PASS 4 (125s) == lustre-rsync-test test 5a: Stop / start lustre_rsync == 08:35:00 (1789130100) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' lustre-MDT0000: Registered changelog user cl9 - open/close 741 (time 1789130119.50 total 10.00 last 74.10) - open/close 1398 (time 1789130129.51 total 20.01 last 65.64) - open/close 1973 (time 1789130139.52 total 30.02 last 57.47) total: 2000 open/close in 30.55 seconds: 65.47 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl9 Starting changelog record: 0 Clear changelog after use: no lustre_rsync halting. Errors: 0 lustre_rsync took 30 seconds Changelog records consumed: 799 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl9 Starting changelog record: 19609 Clear changelog after use: no Errors: 0 lustre_rsync took 121 seconds Changelog records consumed: 3202 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0001: changelog user 'cl9' not found lustre-MDT0000: changelog user 'cl9' not found PASS 5a (248s) == lustre-rsync-test test 5b: Kill / restart lustre_rsync ========================================================== 08:39:08 (1789130348) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' lustre-MDT0000: Registered changelog user cl10 - open/close 710 (time 1789130371.11 total 10.00 last 70.98) - open/close 1438 (time 1789130381.11 total 20.01 last 72.75) total: 2000 open/close in 28.39 seconds: 70.44 ops/second /home/green/git/lustre-release/lustre/tests/lustre-rsync-test.sh: line 538: 19774 Killed $LRSYNC -s $DIR -t $TGT -t $TGT2 -m $MDT0 -u $CL_USER -l $LREPL_LOG -D $LRSYNC_LOG Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl10 Starting changelog record: 23619 Clear changelog after use: no Errors: 0 lustre_rsync took 122 seconds Changelog records consumed: 3192 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0001: changelog user 'cl10' not found lustre-MDT0000: changelog user 'cl10' not found PASS 5b (253s) == lustre-rsync-test test 6: lustre_rsync large no of hard links ========================================================== 08:43:21 (1789130601) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' lustre-MDT0000: Registered changelog user cl11 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl11 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 129 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0001: changelog user 'cl11' not found lustre-MDT0000: changelog user 'cl11' not found PASS 6 (36s) == lustre-rsync-test test 7: lustre_rsync stripesize ===== 08:43:57 (1789130637) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' lustre-MDT0000: Registered changelog user cl12 total: 100 open/close in 1.62 seconds: 61.59 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/lustre/tgt Statuslog: /tmp/lustre_rsync.log Changelog registration: cl12 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 34 seconds Changelog records consumed: 202 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0001: changelog user 'cl12' not found lustre-MDT0000: changelog user 'cl12' not found PASS 7 (74s) == lustre-rsync-test test 8: Replicate multiple file/directory moves ========================================================== 08:45:11 (1789130711) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0000: Registered changelog user cl13 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl13 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 103 seconds Changelog records consumed: 3501 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0001: changelog user 'cl13' not found lustre-MDT0000: changelog user 'cl13' not found PASS 8 (224s) == lustre-rsync-test test 9: Replicate recursive directory removal ========================================================== 08:48:55 (1789130935) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl14 cl14' lustre-MDT0000: Registered changelog user cl14 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl14 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl14 Starting changelog record: 30645 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 lustre-MDT0000: clear the changelog for cl14 of all records lustre-MDT0001: clear the changelog for cl14 of all records lustre-MDT0000: clear the changelog for cl14 of all records lustre-MDT0000: Deregistered changelog user #14 lustre-MDT0001: clear the changelog for cl14 of all records lustre-MDT0001: Deregistered changelog user #14 lustre-MDT0001: changelog user 'cl14' not found lustre-MDT0000: changelog user 'cl14' not found PASS 9 (20s) == lustre-rsync-test test complete, duration 1640 sec ==== 08:49:16 (1789130956) osc.lustre-OST0000-osc-ffff984e0a0d8000.stats_compr= write_pages_compressible: 0 write_pages_incompressible: 0 write_chunks_compressible: 0 write_chunks_incompressible: 0 write_bytes_compr: 0 write_bytes_raw: 0 write_bytes_incompr: 0 read_pages_compressed: 0 read_chunks_compressed: 0 read_bytes_compr: 0 read_bytes_raw: 0 osc.lustre-OST0001-osc-ffff984e0a0d8000.stats_compr= write_pages_compressible: 0 write_pages_incompressible: 0 write_chunks_compressible: 0 write_chunks_incompressible: 0 write_bytes_compr: 0 write_bytes_raw: 0 write_bytes_incompr: 0 read_pages_compressed: 0 read_chunks_compressed: 0 read_bytes_compr: 0 read_bytes_raw: 0 obdfilter.lustre-OST0000.stats_compr= snapshot_time 1789130959.546537324 secs.nsecs start_time 1789129386.529004009 secs.nsecs elapsed_time 1573.017533315 secs.nsecs uncompressed_objects 410 samples [reqs] obdfilter.lustre-OST0001.stats_compr= snapshot_time 1789130959.546597859 secs.nsecs start_time 1789129386.529030160 secs.nsecs elapsed_time 1573.017567699 secs.nsecs uncompressed_objects 384 samples [reqs] === lustre-rsync-test: start cleanup 08:49:19 (1789130959) === === lustre-rsync-test: finish cleanup 08:49:23 (1789130963) ===