-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:43:42 EDT 2025 oleg142-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg142-client.virtnet: executing check_config_client /mnt/lustre oleg142-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg142-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b62ba000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b62ba000.idle_timeout=debug disable quota as required oleg142-server: oleg142-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:52 (1748036632) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:54 EDT 2025 waiting for dbench pid 8740 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 455 31.81 MB/sec warmup 1 sec latency 16.560 ms 2 1063 22.89 MB/sec warmup 2 sec latency 10.836 ms 2 1698 17.48 MB/sec warmup 3 sec latency 10.983 ms 2 2395 13.99 MB/sec warmup 4 sec latency 9.764 ms 2 3354 14.15 MB/sec warmup 5 sec latency 10.943 ms 2 3977 13.35 MB/sec warmup 6 sec latency 10.344 ms 2 4527 11.69 MB/sec warmup 7 sec latency 15.058 ms 2 5190 11.32 MB/sec warmup 8 sec latency 10.921 ms 2 5855 10.42 MB/sec warmup 9 sec latency 10.507 ms 2 6744 10.67 MB/sec warmup 10 sec latency 10.120 ms 2 7441 10.76 MB/sec warmup 11 sec latency 10.404 ms 2 8602 8.95 MB/sec execute 1 sec latency 17.746 ms 2 9247 5.10 MB/sec execute 2 sec latency 11.083 ms 2 10066 7.96 MB/sec execute 3 sec latency 9.494 ms 2 10898 9.09 MB/sec execute 4 sec latency 10.741 ms 2 11419 7.56 MB/sec execute 5 sec latency 13.957 ms 2 12061 7.85 MB/sec execute 6 sec latency 10.019 ms 2 12687 6.89 MB/sec execute 7 sec latency 12.594 ms 2 13573 7.76 MB/sec execute 8 sec latency 9.159 ms 2 14453 8.29 MB/sec execute 9 sec latency 10.653 ms 2 14986 7.60 MB/sec execute 10 sec latency 18.146 ms 2 15653 7.76 MB/sec execute 11 sec latency 18.031 ms 2 16331 7.22 MB/sec execute 12 sec latency 9.063 ms 2 17200 7.72 MB/sec execute 13 sec latency 13.395 ms 2 18067 8.06 MB/sec execute 14 sec latency 9.971 ms 2 18672 7.67 MB/sec execute 15 sec latency 11.925 ms 2 19436 7.75 MB/sec execute 16 sec latency 9.465 ms 2 20209 7.52 MB/sec execute 17 sec latency 8.574 ms 2 21346 8.25 MB/sec execute 18 sec latency 9.005 ms 2 21886 7.99 MB/sec execute 19 sec latency 9.154 ms 2 22539 7.80 MB/sec execute 20 sec latency 13.522 ms 2 23266 7.75 MB/sec execute 21 sec latency 9.312 ms 2 24087 7.90 MB/sec execute 22 sec latency 21.194 ms 2 25107 8.24 MB/sec execute 23 sec latency 9.351 ms 2 25665 7.95 MB/sec execute 24 sec latency 11.747 ms 2 26384 8.01 MB/sec execute 25 sec latency 9.502 ms 2 27200 7.84 MB/sec execute 26 sec latency 8.808 ms 2 28222 8.11 MB/sec execute 27 sec latency 10.848 ms 2 28834 8.15 MB/sec execute 28 sec latency 10.361 ms 2 29395 7.93 MB/sec execute 29 sec latency 15.415 ms 2 30064 7.96 MB/sec execute 30 sec latency 12.084 ms 2 30787 7.81 MB/sec execute 31 sec latency 9.781 ms 2 31774 8.04 MB/sec execute 32 sec latency 9.951 ms 2 32393 8.08 MB/sec execute 33 sec latency 9.898 ms 2 32964 7.89 MB/sec execute 34 sec latency 12.400 ms 2 33656 7.92 MB/sec execute 35 sec latency 10.422 ms 2 34374 7.80 MB/sec execute 36 sec latency 9.450 ms 2 35476 8.14 MB/sec execute 37 sec latency 9.184 ms 2 35990 8.02 MB/sec execute 38 sec latency 9.389 ms 2 36597 7.92 MB/sec execute 39 sec latency 14.229 ms 2 37203 7.88 MB/sec execute 40 sec latency 10.730 ms 2 37896 7.77 MB/sec execute 41 sec latency 10.086 ms 2 38960 8.03 MB/sec execute 42 sec latency 8.486 ms 2 39559 7.98 MB/sec execute 43 sec latency 8.805 ms 2 40219 7.89 MB/sec execute 44 sec latency 12.873 ms 2 40903 7.86 MB/sec execute 45 sec latency 9.646 ms 2 41640 7.80 MB/sec execute 46 sec latency 8.524 ms 2 42689 8.05 MB/sec execute 47 sec latency 10.965 ms 2 43128 7.94 MB/sec execute 48 sec latency 10.644 ms 2 43708 7.86 MB/sec execute 49 sec latency 20.931 ms 2 44355 7.84 MB/sec execute 50 sec latency 10.502 ms 2 45026 7.76 MB/sec execute 51 sec latency 9.134 ms 2 45973 7.89 MB/sec execute 52 sec latency 16.182 ms 2 46555 7.91 MB/sec execute 53 sec latency 14.510 ms 2 47061 7.79 MB/sec execute 54 sec latency 18.618 ms 2 47746 7.82 MB/sec execute 55 sec latency 9.174 ms 2 48482 7.74 MB/sec execute 56 sec latency 9.842 ms 2 49419 7.86 MB/sec execute 57 sec latency 10.762 ms 2 50049 7.89 MB/sec execute 58 sec latency 17.868 ms 2 50562 7.78 MB/sec execute 59 sec latency 14.561 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 14956 4.441 17.861 Close 11004 0.918 2.972 Rename 640 6.586 16.169 Unlink 3004 1.657 18.125 Qpathinfo 13546 1.120 9.442 Qfileinfo 2362 0.122 1.571 Qfsinfo 2486 0.075 1.295 Sfileinfo 1246 3.048 10.655 Find 5236 0.338 7.640 WriteX 7398 0.877 6.505 ReadX 23454 0.012 1.358 LockX 48 0.865 1.099 UnlockX 48 0.913 1.209 Flush 1046 5.450 21.186 Throughput 7.77864 MB/sec 2 clients 2 procs max_latency=21.194 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:45:06 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 9 seconds Changelog records consumed: 20860 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (86s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:45:18 (1748036718) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:20 EDT 2025 waiting for dbench pid 9626 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 477 33.37 MB/sec warmup 1 sec latency 12.837 ms 2 1060 22.89 MB/sec warmup 2 sec latency 12.427 ms 2 1709 17.49 MB/sec warmup 3 sec latency 10.575 ms 2 2394 13.98 MB/sec warmup 4 sec latency 12.338 ms 2 3355 14.15 MB/sec warmup 5 sec latency 11.161 ms 2 3969 13.35 MB/sec warmup 6 sec latency 10.670 ms 2 4493 11.68 MB/sec warmup 7 sec latency 13.823 ms 2 5147 11.32 MB/sec warmup 8 sec latency 10.895 ms 2 5754 10.19 MB/sec warmup 9 sec latency 10.729 ms 2 6567 10.53 MB/sec warmup 10 sec latency 11.969 ms 2 7388 10.70 MB/sec warmup 11 sec latency 12.397 ms 2 8562 9.31 MB/sec execute 1 sec latency 15.961 ms 2 9170 5.26 MB/sec execute 2 sec latency 10.953 ms PIDs: 9626 9627 9628 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 3844 Resuming dbench 2 9468 2.35 MB/sec execute 6 sec latency 3165.300 ms 2 10471 4.30 MB/sec execute 7 sec latency 3165.498 ms 2 11084 4.95 MB/sec execute 8 sec latency 10.068 ms 2 11707 4.83 MB/sec execute 9 sec latency 14.239 ms 2 12417 5.03 MB/sec execute 10 sec latency 10.881 ms 2 13144 5.02 MB/sec execute 11 sec latency 10.251 ms 2 14304 6.31 MB/sec execute 12 sec latency 8.820 ms 2 14878 6.06 MB/sec execute 13 sec latency 14.518 ms 2 15621 6.33 MB/sec execute 14 sec latency 10.022 ms 2 16362 5.99 MB/sec execute 15 sec latency 8.194 ms 2 17465 6.71 MB/sec execute 16 sec latency 8.624 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 24704 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 3093 Wait for dbench to finish 2 17467 6.03 MB/sec execute 17 sec latency 1766.382 ms 2 18189 6.21 MB/sec execute 18 sec latency 1770.838 ms 2 18826 6.10 MB/sec execute 19 sec latency 28.146 ms 2 19539 6.13 MB/sec execute 20 sec latency 13.677 ms 2 20213 6.02 MB/sec execute 21 sec latency 12.431 ms 2 21011 6.31 MB/sec execute 22 sec latency 14.912 ms 2 21548 6.50 MB/sec execute 23 sec latency 12.610 ms 2 21846 6.26 MB/sec execute 24 sec latency 16.786 ms 2 22200 6.06 MB/sec execute 25 sec latency 21.931 ms 2 22525 5.94 MB/sec execute 26 sec latency 17.772 ms 2 22951 5.95 MB/sec execute 27 sec latency 17.585 ms 2 23361 5.77 MB/sec execute 28 sec latency 17.638 ms 2 23778 5.68 MB/sec execute 29 sec latency 17.182 ms 2 24330 5.85 MB/sec execute 30 sec latency 16.282 ms 2 24962 5.99 MB/sec execute 31 sec latency 18.131 ms 2 25237 5.89 MB/sec execute 32 sec latency 18.376 ms 2 25541 5.72 MB/sec execute 33 sec latency 17.894 ms 2 25799 5.60 MB/sec execute 34 sec latency 30.672 ms 2 26149 5.52 MB/sec execute 35 sec latency 16.780 ms 2 26559 5.53 MB/sec execute 36 sec latency 18.733 ms 2 26961 5.41 MB/sec execute 37 sec latency 18.620 ms 2 27434 5.39 MB/sec execute 38 sec latency 16.534 ms 2 28143 5.57 MB/sec execute 39 sec latency 19.350 ms 2 28574 5.61 MB/sec execute 40 sec latency 13.397 ms 2 28850 5.53 MB/sec execute 41 sec latency 18.003 ms 2 29206 5.42 MB/sec execute 42 sec latency 19.053 ms 2 29541 5.37 MB/sec execute 43 sec latency 18.062 ms 2 29917 5.39 MB/sec execute 44 sec latency 18.596 ms 2 30432 5.29 MB/sec execute 45 sec latency 18.568 ms 2 30896 5.27 MB/sec execute 46 sec latency 16.230 ms 2 31434 5.36 MB/sec execute 47 sec latency 17.517 ms 2 32051 5.46 MB/sec execute 48 sec latency 16.835 ms 2 32326 5.41 MB/sec execute 49 sec latency 17.684 ms 2 32697 5.32 MB/sec execute 50 sec latency 19.873 ms 2 33011 5.28 MB/sec execute 51 sec latency 16.760 ms 2 33481 5.30 MB/sec execute 52 sec latency 15.433 ms 2 33795 5.21 MB/sec execute 53 sec latency 18.453 ms 2 34182 5.13 MB/sec execute 54 sec latency 13.942 ms 2 34558 5.12 MB/sec execute 55 sec latency 16.085 ms 2 35314 5.25 MB/sec execute 56 sec latency 16.990 ms 2 35732 5.31 MB/sec execute 57 sec latency 16.715 ms 2 35961 5.23 MB/sec execute 58 sec latency 17.918 ms 2 36297 5.16 MB/sec execute 59 sec latency 19.615 ms 2 cleanup 60 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 9946 6.503 1770.832 Close 7324 1.344 18.213 Rename 422 9.483 19.332 Unlink 2000 2.233 10.105 Qpathinfo 8978 1.916 3165.460 Qfileinfo 1566 0.146 2.319 Qfsinfo 1658 0.112 3.686 Sfileinfo 826 4.343 20.325 Find 3484 1.327 3165.490 WriteX 4942 1.160 8.831 ReadX 15564 0.014 0.149 LockX 32 1.317 2.076 UnlockX 32 1.316 2.129 Flush 712 7.617 30.660 Throughput 5.1579 MB/sec 2 clients 2 procs max_latency=3165.498 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:46:33 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 27796 Clear changelog after use: no Errors: 0 lustre_rsync took 8 seconds Changelog records consumed: 8041 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (86s) == lustre-rsync-test test complete, duration 182 sec ===== 17:46:44 (1748036804)