-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:42:58 EDT 2025 oleg154-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg154-client.virtnet: executing check_config_client /mnt/lustre oleg154-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg154-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b628f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b628f000.idle_timeout=debug disable quota as required oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:07 (1748036587) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:09 EDT 2025 waiting for dbench pid 8769 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 485 33.89 MB/sec warmup 1 sec latency 14.467 ms 2 1118 22.97 MB/sec warmup 2 sec latency 11.337 ms 2 1806 17.55 MB/sec warmup 3 sec latency 10.192 ms 2 2563 14.45 MB/sec warmup 4 sec latency 9.238 ms 2 3685 15.50 MB/sec warmup 5 sec latency 11.311 ms 2 4238 13.43 MB/sec warmup 6 sec latency 14.123 ms 2 4834 12.07 MB/sec warmup 7 sec latency 12.944 ms 2 5539 11.39 MB/sec warmup 8 sec latency 9.958 ms 2 6352 11.35 MB/sec warmup 9 sec latency 12.239 ms 2 7329 11.78 MB/sec warmup 10 sec latency 10.791 ms 2 7843 10.83 MB/sec warmup 11 sec latency 12.777 ms 2 9131 1.06 MB/sec execute 1 sec latency 14.095 ms 2 10002 6.81 MB/sec execute 2 sec latency 8.524 ms 2 10888 9.17 MB/sec execute 3 sec latency 9.521 ms 2 11395 7.21 MB/sec execute 4 sec latency 13.593 ms 2 12061 7.65 MB/sec execute 5 sec latency 9.985 ms 2 12681 6.55 MB/sec execute 6 sec latency 9.324 ms 2 13548 7.41 MB/sec execute 7 sec latency 10.258 ms 2 14436 8.22 MB/sec execute 8 sec latency 10.550 ms 2 14945 7.45 MB/sec execute 9 sec latency 11.569 ms 2 15601 7.65 MB/sec execute 10 sec latency 15.637 ms 2 16207 7.04 MB/sec execute 11 sec latency 9.708 ms 2 16967 7.37 MB/sec execute 12 sec latency 10.630 ms 2 17951 8.00 MB/sec execute 13 sec latency 10.006 ms 2 18464 7.51 MB/sec execute 14 sec latency 13.992 ms 2 19036 7.26 MB/sec execute 15 sec latency 13.480 ms 2 19735 7.23 MB/sec execute 16 sec latency 9.697 ms 2 20501 7.45 MB/sec execute 17 sec latency 9.593 ms 2 21485 7.80 MB/sec execute 18 sec latency 9.866 ms 2 22007 7.55 MB/sec execute 19 sec latency 15.116 ms 2 22564 7.36 MB/sec execute 20 sec latency 9.935 ms 2 23198 7.32 MB/sec execute 21 sec latency 15.438 ms 2 23883 7.22 MB/sec execute 22 sec latency 63.934 ms 2 24937 7.75 MB/sec execute 23 sec latency 11.217 ms 2 25422 7.55 MB/sec execute 24 sec latency 9.819 ms 2 26038 7.41 MB/sec execute 25 sec latency 12.214 ms 2 26680 7.38 MB/sec execute 26 sec latency 15.161 ms 2 27353 7.29 MB/sec execute 27 sec latency 9.211 ms 2 28441 7.72 MB/sec execute 28 sec latency 9.249 ms 2 28911 7.57 MB/sec execute 29 sec latency 10.492 ms 2 29541 7.45 MB/sec execute 30 sec latency 11.364 ms 2 30193 7.42 MB/sec execute 31 sec latency 13.414 ms 2 30868 7.30 MB/sec execute 32 sec latency 9.778 ms 2 31947 7.71 MB/sec execute 33 sec latency 10.772 ms 2 32424 7.58 MB/sec execute 34 sec latency 9.961 ms 2 33002 7.47 MB/sec execute 35 sec latency 16.876 ms 2 33655 7.45 MB/sec execute 36 sec latency 13.855 ms 2 34341 7.34 MB/sec execute 37 sec latency 12.406 ms 2 35294 7.54 MB/sec execute 38 sec latency 9.850 ms 2 35882 7.59 MB/sec execute 39 sec latency 11.338 ms 2 36404 7.43 MB/sec execute 40 sec latency 15.757 ms 2 37030 7.47 MB/sec execute 41 sec latency 11.838 ms 2 37694 7.32 MB/sec execute 42 sec latency 9.790 ms 2 38494 7.47 MB/sec execute 43 sec latency 10.353 ms 2 39318 7.58 MB/sec execute 44 sec latency 11.270 ms 2 39807 7.44 MB/sec execute 45 sec latency 14.532 ms 2 40459 7.48 MB/sec execute 46 sec latency 9.731 ms 2 41062 7.35 MB/sec execute 47 sec latency 10.953 ms 2 41834 7.42 MB/sec execute 48 sec latency 9.715 ms 2 42767 7.55 MB/sec execute 49 sec latency 11.184 ms 2 43271 7.46 MB/sec execute 50 sec latency 21.653 ms 2 43827 7.38 MB/sec execute 51 sec latency 17.053 ms 2 44460 7.37 MB/sec execute 52 sec latency 14.083 ms 2 45193 7.33 MB/sec execute 53 sec latency 9.008 ms 2 46290 7.56 MB/sec execute 54 sec latency 10.904 ms 2 46834 7.47 MB/sec execute 55 sec latency 11.234 ms 2 47439 7.41 MB/sec execute 56 sec latency 12.448 ms 2 48157 7.40 MB/sec execute 57 sec latency 10.692 ms 2 49039 7.49 MB/sec execute 58 sec latency 10.333 ms 2 49942 7.60 MB/sec execute 59 sec latency 9.579 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 14566 4.589 21.640 Close 10674 0.953 4.632 Rename 608 6.793 11.828 Unlink 2950 1.672 3.314 Qpathinfo 13194 1.141 8.220 Qfileinfo 2304 0.123 1.442 Qfsinfo 2408 0.078 1.167 Sfileinfo 1146 3.158 15.623 Find 5086 0.347 7.427 WriteX 7164 0.902 5.817 ReadX 22752 0.012 1.096 LockX 48 0.977 1.350 UnlockX 48 0.959 1.255 Flush 1014 5.602 63.927 Throughput 7.59689 MB/sec 2 clients 2 procs max_latency=63.934 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:21 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 8 seconds Changelog records consumed: 20564 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (84s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:31 (1748036671) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:33 EDT 2025 waiting for dbench pid 9655 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 528 36.50 MB/sec warmup 1 sec latency 10.982 ms 2 1180 23.01 MB/sec warmup 2 sec latency 10.519 ms 2 1856 17.57 MB/sec warmup 3 sec latency 10.302 ms 2 2574 14.47 MB/sec warmup 4 sec latency 10.685 ms 2 3678 15.49 MB/sec warmup 5 sec latency 10.458 ms 2 4227 13.39 MB/sec warmup 6 sec latency 11.520 ms 2 4818 12.06 MB/sec warmup 7 sec latency 12.699 ms 2 5530 11.39 MB/sec warmup 8 sec latency 9.794 ms 2 6326 11.35 MB/sec warmup 9 sec latency 10.540 ms 2 7325 11.78 MB/sec warmup 10 sec latency 9.698 ms 2 7841 10.83 MB/sec warmup 11 sec latency 16.615 ms 2 9123 1.02 MB/sec execute 1 sec latency 14.006 ms 2 9865 5.94 MB/sec execute 2 sec latency 11.505 ms PIDs: 9655 9656 9657 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 4072 Resuming dbench 2 10374 3.29 MB/sec execute 6 sec latency 3205.291 ms 2 11052 4.21 MB/sec execute 7 sec latency 3205.429 ms 2 11641 3.91 MB/sec execute 8 sec latency 14.699 ms 2 12311 4.45 MB/sec execute 9 sec latency 19.500 ms 2 13013 4.34 MB/sec execute 10 sec latency 9.954 ms 2 13920 5.33 MB/sec execute 11 sec latency 11.065 ms 2 14578 5.68 MB/sec execute 12 sec latency 10.721 ms 2 15111 5.35 MB/sec execute 13 sec latency 13.502 ms 2 15773 5.61 MB/sec execute 14 sec latency 15.799 ms 2 16401 5.32 MB/sec execute 15 sec latency 12.509 ms 2 17216 5.84 MB/sec execute 16 sec latency 12.447 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 24636 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 2837 Wait for dbench to finish 2 17217 5.21 MB/sec execute 18 sec latency 1891.665 ms 2 18100 5.63 MB/sec execute 19 sec latency 1891.846 ms 2 18619 5.41 MB/sec execute 20 sec latency 32.643 ms 2 19248 5.58 MB/sec execute 21 sec latency 15.470 ms 2 19886 5.38 MB/sec execute 22 sec latency 11.433 ms 2 20682 5.69 MB/sec execute 23 sec latency 10.577 ms 2 21549 6.04 MB/sec execute 24 sec latency 10.249 ms 2 22057 5.85 MB/sec execute 25 sec latency 14.362 ms 2 22709 5.99 MB/sec execute 26 sec latency 18.210 ms 2 23310 5.79 MB/sec execute 27 sec latency 10.378 ms 2 24103 5.99 MB/sec execute 28 sec latency 9.259 ms 2 25044 6.26 MB/sec execute 29 sec latency 10.160 ms 2 25574 6.14 MB/sec execute 30 sec latency 13.179 ms 2 26177 6.09 MB/sec execute 31 sec latency 10.108 ms 2 26842 6.09 MB/sec execute 32 sec latency 16.240 ms 2 27633 6.24 MB/sec execute 33 sec latency 10.027 ms 2 28557 6.46 MB/sec execute 34 sec latency 10.212 ms 2 29095 6.36 MB/sec execute 35 sec latency 12.894 ms 2 29656 6.29 MB/sec execute 36 sec latency 9.929 ms 2 30327 6.30 MB/sec execute 37 sec latency 12.467 ms 2 31022 6.27 MB/sec execute 38 sec latency 8.574 ms 2 32095 6.61 MB/sec execute 39 sec latency 10.058 ms 2 32581 6.51 MB/sec execute 40 sec latency 10.238 ms 2 33163 6.45 MB/sec execute 41 sec latency 14.434 ms 2 33804 6.46 MB/sec execute 42 sec latency 11.227 ms 2 34523 6.43 MB/sec execute 43 sec latency 12.164 ms 2 35578 6.72 MB/sec execute 44 sec latency 10.411 ms 2 36006 6.64 MB/sec execute 45 sec latency 11.192 ms 2 36599 6.58 MB/sec execute 46 sec latency 12.246 ms 2 37227 6.58 MB/sec execute 47 sec latency 14.222 ms 2 37916 6.52 MB/sec execute 48 sec latency 11.516 ms 2 38894 6.69 MB/sec execute 49 sec latency 11.480 ms 2 39494 6.74 MB/sec execute 50 sec latency 10.106 ms 2 40046 6.65 MB/sec execute 51 sec latency 14.591 ms 2 40749 6.68 MB/sec execute 52 sec latency 9.977 ms 2 41465 6.63 MB/sec execute 53 sec latency 9.783 ms 2 42576 6.89 MB/sec execute 54 sec latency 11.209 ms 2 43103 6.82 MB/sec execute 55 sec latency 9.015 ms 2 43739 6.77 MB/sec execute 56 sec latency 13.810 ms 2 44431 6.77 MB/sec execute 57 sec latency 9.952 ms 2 45194 6.75 MB/sec execute 58 sec latency 9.642 ms 2 46323 6.97 MB/sec execute 59 sec latency 8.497 ms 2 46887 6.91 MB/sec execute 60 sec latency 14.298 ms 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13542 5.310 3205.409 Close 9956 0.951 16.790 Rename 572 6.738 11.420 Unlink 2732 1.686 6.291 Qpathinfo 12272 1.133 8.221 Qfileinfo 2140 0.120 1.307 Qfsinfo 2244 0.077 1.310 Sfileinfo 1100 3.142 18.203 Find 4734 0.328 6.448 WriteX 6678 0.885 5.524 ReadX 21206 0.011 1.393 LockX 44 0.915 1.196 UnlockX 44 0.960 1.262 Flush 948 5.525 19.484 Throughput 6.90505 MB/sec 2 clients 2 procs max_latency=3205.429 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:46 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 27472 Clear changelog after use: no Errors: 0 lustre_rsync took 5 seconds Changelog records consumed: 12505 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (82s) == lustre-rsync-test test complete, duration 175 sec ===== 17:45:53 (1748036753)