-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:43:11 EDT 2025 oleg434-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg434-client.virtnet: executing check_config_client /mnt/lustre oleg434-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg434-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a515800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a515800.idle_timeout=debug disable quota as required oleg434-server: oleg434-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:21 (1748036601) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:23 EDT 2025 waiting for dbench pid 9623 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 666 42.32 MB/sec warmup 1 sec latency 14.152 ms 2 1453 26.02 MB/sec warmup 2 sec latency 8.466 ms 2 2269 17.96 MB/sec warmup 3 sec latency 9.842 ms 2 3504 19.17 MB/sec warmup 4 sec latency 8.111 ms 2 4209 16.07 MB/sec warmup 5 sec latency 8.165 ms 2 4994 15.04 MB/sec warmup 6 sec latency 11.675 ms 2 5819 13.12 MB/sec warmup 7 sec latency 8.378 ms 2 7047 14.37 MB/sec warmup 8 sec latency 8.923 ms 2 7704 13.16 MB/sec warmup 9 sec latency 8.510 ms 2 8505 12.85 MB/sec warmup 10 sec latency 21.989 ms 2 9280 11.82 MB/sec warmup 11 sec latency 9.272 ms 2 11140 9.37 MB/sec execute 1 sec latency 7.997 ms 2 11875 6.80 MB/sec execute 2 sec latency 14.297 ms 2 12676 6.85 MB/sec execute 3 sec latency 9.295 ms 2 13674 8.64 MB/sec execute 4 sec latency 9.228 ms 2 14617 9.52 MB/sec execute 5 sec latency 8.229 ms 2 15327 8.61 MB/sec execute 6 sec latency 16.477 ms 2 16110 8.37 MB/sec execute 7 sec latency 8.672 ms 2 17127 9.09 MB/sec execute 8 sec latency 8.782 ms 2 18110 9.54 MB/sec execute 9 sec latency 8.154 ms 2 18753 8.80 MB/sec execute 10 sec latency 13.134 ms 2 19555 8.78 MB/sec execute 11 sec latency 8.449 ms 2 20392 8.51 MB/sec execute 12 sec latency 10.218 ms 2 21593 9.51 MB/sec execute 13 sec latency 7.746 ms 2 22220 8.97 MB/sec execute 14 sec latency 13.504 ms 2 23019 8.98 MB/sec execute 15 sec latency 8.133 ms 2 23876 8.76 MB/sec execute 16 sec latency 8.126 ms 2 25079 9.52 MB/sec execute 17 sec latency 8.729 ms 2 25694 9.08 MB/sec execute 18 sec latency 22.070 ms 2 26469 9.10 MB/sec execute 19 sec latency 10.760 ms 2 27283 8.83 MB/sec execute 20 sec latency 8.178 ms 2 28486 9.42 MB/sec execute 21 sec latency 10.245 ms 2 29107 9.14 MB/sec execute 22 sec latency 21.826 ms 2 29889 9.17 MB/sec execute 23 sec latency 14.390 ms 2 30684 8.86 MB/sec execute 24 sec latency 8.769 ms 2 31824 9.22 MB/sec execute 25 sec latency 8.481 ms 2 32532 9.19 MB/sec execute 26 sec latency 8.185 ms 2 33289 9.08 MB/sec execute 27 sec latency 12.195 ms 2 34125 8.94 MB/sec execute 28 sec latency 9.921 ms 2 35258 9.23 MB/sec execute 29 sec latency 7.998 ms 2 35987 9.24 MB/sec execute 30 sec latency 8.225 ms 2 36723 9.08 MB/sec execute 31 sec latency 13.673 ms 2 37568 9.02 MB/sec execute 32 sec latency 7.924 ms 2 38529 9.16 MB/sec execute 33 sec latency 10.095 ms 2 39470 9.28 MB/sec execute 34 sec latency 8.348 ms 2 40179 9.13 MB/sec execute 35 sec latency 13.500 ms 2 40978 9.07 MB/sec execute 36 sec latency 9.190 ms 2 41990 9.20 MB/sec execute 37 sec latency 8.453 ms 2 42953 9.31 MB/sec execute 38 sec latency 7.943 ms 2 43597 9.12 MB/sec execute 39 sec latency 16.134 ms 2 44433 9.11 MB/sec execute 40 sec latency 8.005 ms 2 45477 9.17 MB/sec execute 41 sec latency 8.517 ms 2 46466 9.32 MB/sec execute 42 sec latency 8.163 ms 2 47126 9.15 MB/sec execute 43 sec latency 13.281 ms 2 47960 9.15 MB/sec execute 44 sec latency 7.937 ms 2 48936 9.20 MB/sec execute 45 sec latency 9.249 ms 2 50027 9.35 MB/sec execute 46 sec latency 7.553 ms 2 50698 9.20 MB/sec execute 47 sec latency 14.810 ms 2 51525 9.19 MB/sec execute 48 sec latency 8.044 ms 2 52585 9.26 MB/sec execute 49 sec latency 7.700 ms 2 53598 9.37 MB/sec execute 50 sec latency 8.243 ms 2 54322 9.26 MB/sec execute 51 sec latency 11.836 ms 2 55151 9.22 MB/sec execute 52 sec latency 12.529 ms 2 56219 9.31 MB/sec execute 53 sec latency 7.946 ms 2 57186 9.38 MB/sec execute 54 sec latency 7.827 ms 2 57914 9.28 MB/sec execute 55 sec latency 12.976 ms 2 58751 9.24 MB/sec execute 56 sec latency 13.732 ms 2 59772 9.33 MB/sec execute 57 sec latency 9.264 ms 2 60739 9.39 MB/sec execute 58 sec latency 7.745 ms 2 61476 9.30 MB/sec execute 59 sec latency 9.300 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 18110 3.679 15.691 Close 13286 0.754 4.220 Rename 758 5.426 8.759 Unlink 3676 1.346 2.557 Qpathinfo 16364 0.937 11.435 Qfileinfo 2818 0.105 1.140 Qfsinfo 2982 0.062 0.879 Sfileinfo 1468 2.582 16.470 Find 6242 0.291 5.228 WriteX 8832 0.728 3.814 ReadX 27882 0.010 0.959 LockX 56 0.756 1.710 UnlockX 56 0.770 0.988 Flush 1264 4.415 22.065 Throughput 9.30312 MB/sec 2 clients 2 procs max_latency=22.070 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:35 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 9 seconds Changelog records consumed: 25382 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0001: changelog user 'cl1' not found lustre-MDT0000: changelog user 'cl1' not found PASS 2a (86s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:47 (1748036687) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:49 EDT 2025 waiting for dbench pid 10594 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 674 42.35 MB/sec warmup 1 sec latency 11.071 ms 2 1455 26.02 MB/sec warmup 2 sec latency 8.739 ms 2 2338 18.55 MB/sec warmup 3 sec latency 9.583 ms 2 3578 19.22 MB/sec warmup 4 sec latency 8.971 ms 2 4292 16.16 MB/sec warmup 5 sec latency 11.132 ms 2 5122 15.07 MB/sec warmup 6 sec latency 12.665 ms 2 5966 13.47 MB/sec warmup 7 sec latency 8.784 ms 2 7212 14.46 MB/sec warmup 8 sec latency 8.027 ms 2 7889 13.24 MB/sec warmup 9 sec latency 12.509 ms 2 8682 12.87 MB/sec warmup 10 sec latency 12.759 ms 2 9521 12.06 MB/sec warmup 11 sec latency 8.671 ms 2 11383 3.54 MB/sec execute 1 sec latency 11.279 ms 2 12194 6.57 MB/sec execute 2 sec latency 14.425 ms PIDs: 10594 10595 10596 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 5080 Resuming dbench 2 12562 2.57 MB/sec execute 5 sec latency 2877.692 ms 2 13584 4.38 MB/sec execute 6 sec latency 2877.733 ms 2 14562 5.57 MB/sec execute 7 sec latency 9.456 ms 2 15205 5.17 MB/sec execute 8 sec latency 14.254 ms 2 16006 5.53 MB/sec execute 9 sec latency 15.673 ms 2 16944 6.08 MB/sec execute 10 sec latency 9.275 ms 2 18011 6.92 MB/sec execute 11 sec latency 9.168 ms 2 18658 6.53 MB/sec execute 12 sec latency 10.901 ms 2 19444 6.72 MB/sec execute 13 sec latency 14.120 ms 2 20272 6.61 MB/sec execute 14 sec latency 9.054 ms 2 21495 7.49 MB/sec execute 15 sec latency 8.550 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 30462 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 3493 Wait for dbench to finish 2 21499 6.67 MB/sec execute 17 sec latency 1929.960 ms 2 22156 6.49 MB/sec execute 18 sec latency 1929.968 ms 2 22946 6.63 MB/sec execute 19 sec latency 8.490 ms 2 23767 6.50 MB/sec execute 20 sec latency 8.162 ms 2 24997 7.19 MB/sec execute 21 sec latency 8.208 ms 2 25616 7.02 MB/sec execute 22 sec latency 11.631 ms 2 26363 7.13 MB/sec execute 23 sec latency 21.077 ms 2 27210 6.98 MB/sec execute 24 sec latency 8.832 ms 2 28400 7.54 MB/sec execute 25 sec latency 9.059 ms 2 29069 7.39 MB/sec execute 26 sec latency 8.088 ms 2 29843 7.48 MB/sec execute 27 sec latency 13.044 ms 2 30610 7.27 MB/sec execute 28 sec latency 8.432 ms 2 31767 7.61 MB/sec execute 29 sec latency 8.141 ms 2 32474 7.67 MB/sec execute 30 sec latency 7.925 ms 2 33215 7.56 MB/sec execute 31 sec latency 13.875 ms 2 34055 7.55 MB/sec execute 32 sec latency 10.057 ms 2 35171 7.78 MB/sec execute 33 sec latency 8.926 ms 2 35958 7.89 MB/sec execute 34 sec latency 8.528 ms 2 36672 7.78 MB/sec execute 35 sec latency 12.954 ms 2 37489 7.76 MB/sec execute 36 sec latency 7.899 ms 2 38497 7.93 MB/sec execute 37 sec latency 8.510 ms 2 39450 8.06 MB/sec execute 38 sec latency 8.035 ms 2 40163 7.96 MB/sec execute 39 sec latency 12.633 ms 2 40966 7.93 MB/sec execute 40 sec latency 8.877 ms 2 41958 8.05 MB/sec execute 41 sec latency 8.158 ms 2 42926 8.19 MB/sec execute 42 sec latency 8.518 ms 2 43590 8.06 MB/sec execute 43 sec latency 12.323 ms 2 44427 8.08 MB/sec execute 44 sec latency 7.858 ms 2 45490 8.19 MB/sec execute 45 sec latency 9.109 ms 2 46511 8.32 MB/sec execute 46 sec latency 7.869 ms 2 47233 8.23 MB/sec execute 47 sec latency 19.873 ms 2 48024 8.20 MB/sec execute 48 sec latency 18.335 ms 2 48654 8.13 MB/sec execute 49 sec latency 14.778 ms 2 49532 8.24 MB/sec execute 50 sec latency 15.119 ms 2 49983 8.24 MB/sec execute 51 sec latency 17.310 ms 2 50255 8.10 MB/sec execute 52 sec latency 16.548 ms 2 50610 7.98 MB/sec execute 53 sec latency 26.414 ms 2 51358 7.99 MB/sec execute 54 sec latency 14.324 ms 2 52092 7.91 MB/sec execute 55 sec latency 9.763 ms 2 52624 7.97 MB/sec execute 56 sec latency 16.074 ms 2 53284 8.00 MB/sec execute 57 sec latency 15.667 ms 2 53622 7.92 MB/sec execute 58 sec latency 16.910 ms 2 54064 7.80 MB/sec execute 59 sec latency 19.540 ms 2 cleanup 60 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 15370 4.257 1929.956 Close 11230 1.332 2877.720 Rename 648 6.066 15.903 Unlink 3150 1.505 4.260 Qpathinfo 13856 1.014 10.362 Qfileinfo 2384 0.112 1.128 Qfsinfo 2540 0.071 1.692 Sfileinfo 1268 2.767 14.419 Find 5320 0.310 9.933 WriteX 7532 0.788 6.718 ReadX 23670 0.010 0.607 LockX 48 0.841 1.673 UnlockX 48 0.848 1.723 Flush 1082 4.765 26.403 Throughput 7.80459 MB/sec 2 clients 2 procs max_latency=2877.733 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:46:02 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 33954 Clear changelog after use: no Errors: 0 lustre_rsync took 9 seconds Changelog records consumed: 13749 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0001: changelog user 'cl2' not found lustre-MDT0000: changelog user 'cl2' not found PASS 2b (88s) == lustre-rsync-test test complete, duration 184 sec ===== 17:46:15 (1748036775)