-----============= acceptance-small: lustre-rsync-test ============----- Mon Sep 14 02:19:29 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg328-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex': No such file or directory excepting tests: 2b === lustre-rsync-test: start setup 02:19:39 (1789366779) === oleg328-client.virtnet: executing check_config_client /mnt/lustre oleg328-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg328-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8c14d07b8000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8c14d07b8000.idle_timeout=debug disable quota as required oleg328-server: oleg328-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === lustre-rsync-test: finish setup 02:20:04 (1789366804) === nobody:x:65534: == lustre-rsync-test test 1A: Simple Replication ========= 02:20:06 (1789366806) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000: Registered changelog user cl1 Replication #1 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 26 Replication #2 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 26 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0001: changelog user 'cl1' not found lustre-MDT0000: changelog user 'cl1' not found PASS 1A (14s) == lustre-rsync-test test 1a: Replicate create/delete operations in ROOT directory ========================================================== 02:20:21 (1789366821) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' lustre-MDT0000: Registered changelog user cl2 Replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 11 File: /mnt/target/root-dir Size: 0 Blocks: 0 IO Block: 4096 directory Device: 2bh/43d Inode: 49384 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-14 02:20:25.000000000 -0400 Modify: 2026-09-14 02:20:25.000000000 -0400 Change: 2026-09-14 02:20:25.985000000 -0400 Birth: - File: /mnt/target/root-file Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 2bh/43d Inode: 49385 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-14 02:20:25.000000000 -0400 Modify: 2026-09-14 02:20:25.000000000 -0400 Change: 2026-09-14 02:20:26.014000000 -0400 Birth: - stat: cannot statx '/mnt/target/root-file2': No such file or directory File: /mnt/target/root-file3 Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 2bh/43d Inode: 49386 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-14 02:20:26.000000000 -0400 Modify: 2026-09-14 02:20:25.000000000 -0400 Change: 2026-09-14 02:20:26.056000000 -0400 Birth: - stat: cannot statx '/mnt/target/root-dir1': No such file or directory stat: cannot statx '/mnt/target/root-file4': No such file or directory lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0001: changelog user 'cl2' not found lustre-MDT0000: changelog user 'cl2' not found PASS 1a (14s) == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 02:20:34 (1789366834) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' lustre-MDT0000: Registered changelog user cl3 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Mon Sep 14 02:20:40 EDT 2026 waiting for dbench pid 12498 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 133 11.78 MB/sec warmup 1 sec latency 74.683 ms 2 223 8.61 MB/sec warmup 2 sec latency 162.291 ms 2 345 8.24 MB/sec warmup 3 sec latency 53.005 ms 2 485 8.45 MB/sec warmup 4 sec latency 52.583 ms 2 681 8.45 MB/sec warmup 5 sec latency 53.759 ms 2 821 7.15 MB/sec warmup 6 sec latency 38.622 ms 2 922 6.21 MB/sec warmup 7 sec latency 53.073 ms 2 1075 5.72 MB/sec warmup 8 sec latency 55.690 ms 2 1207 5.12 MB/sec warmup 9 sec latency 48.953 ms 2 1431 5.20 MB/sec warmup 10 sec latency 40.953 ms 2 1579 4.74 MB/sec warmup 11 sec latency 38.564 ms 2 1855 0.27 MB/sec execute 1 sec latency 71.876 ms 2 2007 0.29 MB/sec execute 2 sec latency 52.253 ms 2 2161 0.33 MB/sec execute 3 sec latency 47.665 ms 2 2344 0.82 MB/sec execute 4 sec latency 56.251 ms 2 2428 0.75 MB/sec execute 5 sec latency 58.495 ms 2 2587 0.92 MB/sec execute 6 sec latency 46.755 ms 2 2919 2.07 MB/sec execute 7 sec latency 52.825 ms 2 3199 2.01 MB/sec execute 8 sec latency 51.405 ms 2 3457 2.41 MB/sec execute 9 sec latency 51.454 ms 2 3655 2.49 MB/sec execute 10 sec latency 59.055 ms 2 3792 2.46 MB/sec execute 11 sec latency 47.862 ms 2 3905 2.30 MB/sec execute 12 sec latency 51.810 ms 2 4003 2.13 MB/sec execute 13 sec latency 49.748 ms 2 4124 1.98 MB/sec execute 14 sec latency 77.004 ms 2 4264 1.88 MB/sec execute 15 sec latency 70.449 ms 2 4356 1.79 MB/sec execute 16 sec latency 54.847 ms 2 4470 1.71 MB/sec execute 17 sec latency 73.057 ms 2 4586 1.72 MB/sec execute 18 sec latency 42.989 ms 2 4740 1.67 MB/sec execute 19 sec latency 49.052 ms 2 4953 1.89 MB/sec execute 20 sec latency 39.740 ms 2 5043 1.80 MB/sec execute 21 sec latency 54.262 ms 2 5161 1.73 MB/sec execute 22 sec latency 58.626 ms 2 5276 1.66 MB/sec execute 23 sec latency 76.818 ms 2 5370 1.60 MB/sec execute 24 sec latency 49.412 ms 2 5533 1.55 MB/sec execute 25 sec latency 43.953 ms 2 5645 1.50 MB/sec execute 26 sec latency 103.497 ms 2 5779 1.46 MB/sec execute 27 sec latency 51.324 ms 2 5927 1.48 MB/sec execute 28 sec latency 70.464 ms 2 6041 1.49 MB/sec execute 29 sec latency 42.518 ms 2 6324 1.66 MB/sec execute 30 sec latency 48.800 ms 2 6557 1.70 MB/sec execute 31 sec latency 42.994 ms 2 6914 1.77 MB/sec execute 32 sec latency 57.257 ms 2 7164 1.91 MB/sec execute 33 sec latency 43.728 ms 2 7305 1.92 MB/sec execute 34 sec latency 61.140 ms 2 7428 1.86 MB/sec execute 35 sec latency 45.452 ms 2 7560 1.83 MB/sec execute 36 sec latency 41.079 ms 2 7692 1.78 MB/sec execute 37 sec latency 99.852 ms 2 7823 1.75 MB/sec execute 38 sec latency 75.337 ms 2 7909 1.72 MB/sec execute 39 sec latency 69.291 ms 2 8020 1.69 MB/sec execute 40 sec latency 43.624 ms 2 8103 1.65 MB/sec execute 41 sec latency 56.718 ms 2 8253 1.67 MB/sec execute 42 sec latency 47.727 ms 2 8354 1.63 MB/sec execute 43 sec latency 72.653 ms 2 8535 1.73 MB/sec execute 44 sec latency 50.482 ms 2 8657 1.69 MB/sec execute 45 sec latency 58.856 ms 2 8748 1.66 MB/sec execute 46 sec latency 51.761 ms 2 8828 1.63 MB/sec execute 47 sec latency 68.663 ms 2 8933 1.60 MB/sec execute 48 sec latency 48.796 ms 2 9086 1.57 MB/sec execute 49 sec latency 47.837 ms 2 9180 1.54 MB/sec execute 50 sec latency 67.081 ms 2 9286 1.52 MB/sec execute 51 sec latency 66.180 ms 2 9461 1.53 MB/sec execute 52 sec latency 58.043 ms 2 9542 1.51 MB/sec execute 53 sec latency 58.612 ms 2 9675 1.52 MB/sec execute 54 sec latency 49.614 ms 2 9927 1.60 MB/sec execute 55 sec latency 52.027 ms 2 10109 1.63 MB/sec execute 56 sec latency 58.645 ms 2 10451 1.66 MB/sec execute 57 sec latency 62.780 ms 2 10648 1.74 MB/sec execute 58 sec latency 53.434 ms 2 10772 1.72 MB/sec execute 59 sec latency 48.312 ms 2 cleanup 60 sec 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 3057 21.217 103.482 Close 2292 5.031 28.328 Rename 134 31.870 76.797 Unlink 588 7.613 32.174 Qpathinfo 2882 5.517 51.684 Qfileinfo 517 0.447 8.405 Qfsinfo 530 0.271 19.412 Sfileinfo 232 22.917 58.481 Find 1124 1.353 40.600 WriteX 1580 3.780 46.068 ReadX 5239 0.031 5.987 LockX 12 7.080 13.902 UnlockX 12 7.727 14.377 Flush 204 25.798 75.321 Throughput 1.71964 MB/sec 2 clients 2 procs max_latency=103.497 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Mon Sep 14 02:21:54 EDT 2026 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory 'clients/client1/~dmtmp/WORDPRO' removed directory 'clients/client1/~dmtmp/PWRPNT' removed directory 'clients/client1/~dmtmp/EXCEL' removed directory 'clients/client1/~dmtmp/PARADOX' removed directory 'clients/client1/~dmtmp/SEED' removed directory 'clients/client1/~dmtmp/PM' removed directory 'clients/client1/~dmtmp/WORD' removed directory 'clients/client1/~dmtmp/ACCESS' removed directory 'clients/client1/~dmtmp/COREL' removed directory 'clients/client1/~dmtmp' removed directory 'clients/client1' removed directory 'clients/client0/~dmtmp/WORDPRO' removed directory 'clients/client0/~dmtmp/PWRPNT' removed directory 'clients/client0/~dmtmp/EXCEL' removed directory 'clients/client0/~dmtmp/PARADOX' removed directory 'clients/client0/~dmtmp/SEED' removed directory 'clients/client0/~dmtmp/PM' removed directory 'clients/client0/~dmtmp/WORD' removed directory 'clients/client0/~dmtmp/ACCESS' removed directory 'clients/client0/~dmtmp/COREL' removed directory 'clients/client0/~dmtmp' removed directory 'clients/client0' removed directory 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl3 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 7 seconds Changelog records consumed: 4498 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0001: changelog user 'cl3' not found lustre-MDT0000: changelog user 'cl3' not found PASS 2a (97s) SKIP: lustre-rsync-test test_2b skipping ALWAYS excluded test 2b == lustre-rsync-test test 2c: Replicate files while dbench is running. ========================================================== 02:22:12 (1789366932) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' lustre-MDT0000: Registered changelog user cl4 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2c.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2c.lustre-rsync-test at Mon Sep 14 02:22:19 EDT 2026 waiting for dbench pid 13517 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 147 12.50 MB/sec warmup 1 sec latency 68.460 ms 2 231 8.86 MB/sec warmup 2 sec latency 87.822 ms 2 340 8.16 MB/sec warmup 3 sec latency 71.195 ms 2 443 7.78 MB/sec warmup 4 sec latency 71.795 ms 2 607 8.38 MB/sec warmup 5 sec latency 66.568 ms 2 730 7.09 MB/sec warmup 6 sec latency 54.308 ms 2 860 6.16 MB/sec warmup 7 sec latency 44.550 ms 2 953 5.44 MB/sec warmup 8 sec latency 84.534 ms Running lustre_rsync 2 1084 5.09 MB/sec warmup 9 sec latency 42.875 ms 2 1214 4.61 MB/sec warmup 10 sec latency 44.586 ms 2 1407 4.72 MB/sec warmup 11 sec latency 54.130 ms 2 1653 0.31 MB/sec execute 1 sec latency 60.870 ms 2 1771 0.23 MB/sec execute 2 sec latency 61.535 ms 2 1906 0.26 MB/sec execute 3 sec latency 48.234 ms 2 2024 0.25 MB/sec execute 4 sec latency 50.461 ms 2 2168 0.27 MB/sec execute 5 sec latency 57.211 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 9 seconds Changelog records consumed: 430 2 2352 0.61 MB/sec execute 6 sec latency 60.670 ms 2 2450 0.59 MB/sec execute 7 sec latency 61.119 ms 2 2606 0.74 MB/sec execute 8 sec latency 43.881 ms 2 2988 1.66 MB/sec execute 9 sec latency 45.596 ms 2 3292 1.86 MB/sec execute 10 sec latency 59.883 ms 13517 13518 13519 Running lustre_rsync 2 3546 2.24 MB/sec execute 11 sec latency 53.353 ms 2 3684 2.11 MB/sec execute 12 sec latency 46.086 ms 2 3814 2.11 MB/sec execute 13 sec latency 48.866 ms 2 3945 2.00 MB/sec execute 14 sec latency 38.066 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 4968 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 831 2 4056 1.87 MB/sec execute 15 sec latency 101.648 ms 2 4200 1.76 MB/sec execute 16 sec latency 56.668 ms 2 4306 1.69 MB/sec execute 17 sec latency 89.169 ms 2 4432 1.63 MB/sec execute 18 sec latency 52.576 ms 2 4535 1.57 MB/sec execute 19 sec latency 120.231 ms 13517 13518 13519 Running lustre_rsync 2 4643 1.60 MB/sec execute 20 sec latency 74.681 ms 2 4748 1.53 MB/sec execute 21 sec latency 57.270 ms 2 4906 1.62 MB/sec execute 22 sec latency 38.760 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 5798 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 608 2 5048 1.66 MB/sec execute 23 sec latency 76.967 ms 2 5146 1.60 MB/sec execute 24 sec latency 96.013 ms 2 5266 1.54 MB/sec execute 25 sec latency 53.343 ms 2 5376 1.49 MB/sec execute 26 sec latency 42.275 ms 2 5534 1.44 MB/sec execute 27 sec latency 59.399 ms 13517 13518 13519 Running lustre_rsync 2 5675 1.41 MB/sec execute 28 sec latency 51.705 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 6405 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 408 2 5807 1.37 MB/sec execute 29 sec latency 51.824 ms 2 5955 1.40 MB/sec execute 30 sec latency 51.287 ms 2 6091 1.41 MB/sec execute 31 sec latency 65.920 ms 2 6386 1.57 MB/sec execute 32 sec latency 53.082 ms 2 6567 1.61 MB/sec execute 33 sec latency 61.416 ms 13517 13518 13519 Running lustre_rsync 2 6898 1.67 MB/sec execute 34 sec latency 49.556 ms 2 7104 1.80 MB/sec execute 35 sec latency 47.199 ms 2 7234 1.77 MB/sec execute 36 sec latency 45.631 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 6812 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 369 2 7357 1.77 MB/sec execute 37 sec latency 51.819 ms 2 7473 1.74 MB/sec execute 38 sec latency 44.731 ms 2 7589 1.70 MB/sec execute 39 sec latency 84.001 ms 2 7722 1.66 MB/sec execute 40 sec latency 43.883 ms 2 7834 1.63 MB/sec execute 41 sec latency 87.572 ms 13517 13518 13519 Running lustre_rsync 2 7933 1.60 MB/sec execute 42 sec latency 62.552 ms 2 8037 1.58 MB/sec execute 43 sec latency 54.346 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 7180 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 541 2 8191 1.59 MB/sec execute 44 sec latency 50.952 ms 2 8344 1.57 MB/sec execute 45 sec latency 40.378 ms 2 8583 1.66 MB/sec execute 46 sec latency 34.667 ms 2 8728 1.63 MB/sec execute 47 sec latency 37.476 ms 2 8864 1.60 MB/sec execute 48 sec latency 39.977 ms 13517 13518 13519 Running lustre_rsync 2 8970 1.57 MB/sec execute 49 sec latency 41.854 ms 2 9122 1.55 MB/sec execute 50 sec latency 53.971 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 7720 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 449 2 9284 1.53 MB/sec execute 51 sec latency 37.724 ms 2 9480 1.54 MB/sec execute 52 sec latency 44.717 ms 2 9594 1.54 MB/sec execute 53 sec latency 55.869 ms 2 9868 1.63 MB/sec execute 54 sec latency 47.017 ms 2 10109 1.66 MB/sec execute 55 sec latency 48.366 ms 13517 13518 13519 Running lustre_rsync 2 10386 1.70 MB/sec execute 56 sec latency 56.117 ms 2 10619 1.77 MB/sec execute 57 sec latency 55.033 ms 2 10748 1.75 MB/sec execute 58 sec latency 45.022 ms Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 8168 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 438 2 10889 1.76 MB/sec execute 59 sec latency 54.195 ms 2 cleanup 60 sec 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 3244 20.315 120.217 Close 2406 4.750 28.566 Rename 136 30.538 65.898 Unlink 620 7.879 24.230 Qpathinfo 3010 5.267 47.131 Qfileinfo 534 0.418 11.286 Qfsinfo 540 0.246 14.314 Sfileinfo 232 20.830 74.668 Find 1152 1.254 38.358 WriteX 1628 3.376 35.791 ReadX 5296 0.027 2.476 LockX 12 4.990 7.368 UnlockX 12 3.234 5.634 Flush 208 25.335 89.154 Throughput 1.75916 MB/sec 2 clients 2 procs max_latency=120.231 ms stopping dbench in /mnt/lustre/d2c.lustre-rsync-test at Mon Sep 14 02:23:32 EDT 2026 with return code 0 clean dbench files in /mnt/lustre/d2c.lustre-rsync-test /mnt/lustre/d2c.lustre-rsync-test /mnt/lustre/d2c.lustre-rsync-test removed directory 'clients/client1/~dmtmp/WORDPRO' removed directory 'clients/client1/~dmtmp/PWRPNT' removed directory 'clients/client1/~dmtmp/EXCEL' removed directory 'clients/client1/~dmtmp/PARADOX' removed directory 'clients/client1/~dmtmp/SEED' removed directory 'clients/client1/~dmtmp/PM' removed directory 'clients/client1/~dmtmp/WORD' removed directory 'clients/client1/~dmtmp/ACCESS' removed directory 'clients/client1/~dmtmp/COREL' removed directory 'clients/client1/~dmtmp' removed directory 'clients/client1' removed directory 'clients/client0/~dmtmp/WORDPRO' removed directory 'clients/client0/~dmtmp/PWRPNT' removed directory 'clients/client0/~dmtmp/EXCEL' removed directory 'clients/client0/~dmtmp/PARADOX' removed directory 'clients/client0/~dmtmp/SEED' removed directory 'clients/client0/~dmtmp/PM' removed directory 'clients/client0/~dmtmp/WORD' removed directory 'clients/client0/~dmtmp/ACCESS' removed directory 'clients/client0/~dmtmp/COREL' removed directory 'clients/client0/~dmtmp' removed directory 'clients/client0' removed directory 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2c.lustre-rsync-test dbench successfully finished Running lustre_rsync Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 8605 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 530 Running final lustre_rsync Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl4 Starting changelog record: 9134 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 1 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0001: changelog user 'cl4' not found lustre-MDT0000: changelog user 'cl4' not found PASS 2c (99s) == lustre-rsync-test test 3a: Replicate files created by createmany ========================================================== 02:23:51 (1789367031) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' lustre-MDT0000: Registered changelog user cl5 total: 1000 open/close in 8.70 seconds: 114.96 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl5 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 47 seconds Changelog records consumed: 2000 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0001: changelog user 'cl5' not found lustre-MDT0000: changelog user 'cl5' not found PASS 3a (82s) == lustre-rsync-test test 3b: Replicate files created by writemany ========================================================== 02:25:13 (1789367113) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' lustre-MDT0000: Registered changelog user cl6 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl6 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 46 seconds Changelog records consumed: 14910 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0001: changelog user 'cl6' not found lustre-MDT0000: changelog user 'cl6' not found PASS 3b (119s) == lustre-rsync-test test 3c: Replicate files created by createmany/unlinkmany ========================================================== 02:27:12 (1789367232) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' lustre-MDT0000: Registered changelog user cl7 total: 1000 open/close in 8.14 seconds: 122.89 ops/second - unlinked 0 (time 1789367247 ; total 0 ; last 0) total: 1000 unlinks in 4 seconds: 250.000000 unlinks/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl7 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 10 seconds Changelog records consumed: 3000 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0001: changelog user 'cl7' not found lustre-MDT0000: changelog user 'cl7' not found PASS 3c (36s) == lustre-rsync-test test 4: Replicate files created by iozone ========================================================== 02:27:48 (1789367268) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lustre-MDT0000: Registered changelog user cl8 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none /mnt/lustre/d4.lustre-rsync-test/d0.iozone-oleg328-client.virtnet stripe_count: 1 stripe_size: 4194304 pattern: stripe_offset: -1 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl8 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 582 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none /mnt/lustre/d4.lustre-rsync-test/d0.iozone-oleg328-client.virtnet stripe_count: 1 stripe_size: 4194304 pattern: stripe_offset: -1 /home/green/git/lustre-release/lustre/tests/lustre-rsync-test.sh: line 448: 17391 Killed END_RUN_FILE=${DIR}/$tdir/run LOAD_PID_FILE=${DIR}/$tdir/pid MOUNT=${DIR}/$tdir RECOVERY_SCALE_ENABLE_REMOTE_DIRS=false RECOVERY_SCALE_ENABLE_STRIPED_DIRS=false run_iozone.sh Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl8 Starting changelog record: 29626 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 555 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0001: changelog user 'cl8' not found lustre-MDT0000: changelog user 'cl8' not found PASS 4 (103s) == lustre-rsync-test test 5a: Stop / start lustre_rsync == 02:29:31 (1789367371) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' lustre-MDT0000: Registered changelog user cl9 - open/close 1385 (time 1789367386.61 total 10.01 last 138.33) total: 2000 open/close in 14.48 seconds: 138.13 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl9 Starting changelog record: 0 Clear changelog after use: no lustre_rsync halting. Errors: 0 lustre_rsync took 31 seconds Changelog records consumed: 1767 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl9 Starting changelog record: 31948 Clear changelog after use: no Errors: 0 lustre_rsync took 38 seconds Changelog records consumed: 2234 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0001: changelog user 'cl9' not found lustre-MDT0000: changelog user 'cl9' not found PASS 5a (115s) == lustre-rsync-test test 5b: Kill / restart lustre_rsync ========================================================== 02:31:26 (1789367486) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' lustre-MDT0000: Registered changelog user cl10 - open/close 1448 (time 1789367500.71 total 10.01 last 144.72) total: 2000 open/close in 14.25 seconds: 140.31 ops/second /home/green/git/lustre-release/lustre/tests/lustre-rsync-test.sh: line 538: 19447 Killed $LRSYNC -s $DIR -t $TGT -t $TGT2 -m $MDT0 -u $CL_USER -l $LREPL_LOG -D $LRSYNC_LOG Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl10 Starting changelog record: 36000 Clear changelog after use: no Errors: 0 lustre_rsync took 33 seconds Changelog records consumed: 2182 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0001: changelog user 'cl10' not found lustre-MDT0000: changelog user 'cl10' not found PASS 5b (107s) == lustre-rsync-test test 6: lustre_rsync large no of hard links ========================================================== 02:33:13 (1789367593) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' lustre-MDT0000: Registered changelog user cl11 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Target: /mnt/target2 Statuslog: /tmp/lustre_rsync.log Changelog registration: cl11 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 129 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0001: changelog user 'cl11' not found lustre-MDT0000: changelog user 'cl11' not found PASS 6 (13s) == lustre-rsync-test test 7: lustre_rsync stripesize ===== 02:33:26 (1789367606) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' lustre-MDT0000: Registered changelog user cl12 total: 100 open/close in 0.58 seconds: 173.24 ops/second Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/lustre/tgt Statuslog: /tmp/lustre_rsync.log Changelog registration: cl12 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 12 seconds Changelog records consumed: 202 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0001: changelog user 'cl12' not found lustre-MDT0000: changelog user 'cl12' not found PASS 7 (25s) == lustre-rsync-test test 8: Replicate multiple file/directory moves ========================================================== 02:33:52 (1789367632) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0000: Registered changelog user cl13 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl13 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 42 seconds Changelog records consumed: 3501 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0001: changelog user 'cl13' not found lustre-MDT0000: changelog user 'cl13' not found PASS 8 (85s) == lustre-rsync-test test 9: Replicate recursive directory removal ========================================================== 02:35:17 (1789367717) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl14 cl14' lustre-MDT0000: Registered changelog user cl14 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl14 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /mnt/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl14 Starting changelog record: 42016 Clear changelog after use: no Errors: 0 lustre_rsync took 0 seconds Changelog records consumed: 3 lustre-MDT0000: clear the changelog for cl14 of all records lustre-MDT0001: clear the changelog for cl14 of all records lustre-MDT0000: clear the changelog for cl14 of all records lustre-MDT0000: Deregistered changelog user #14 lustre-MDT0001: clear the changelog for cl14 of all records lustre-MDT0001: Deregistered changelog user #14 lustre-MDT0001: changelog user 'cl14' not found lustre-MDT0000: changelog user 'cl14' not found PASS 9 (8s) == lustre-rsync-test test complete, duration 955 sec ===== 02:35:25 (1789367725) === lustre-rsync-test: start cleanup 02:35:26 (1789367726) === === lustre-rsync-test: finish cleanup 02:35:27 (1789367727) ===