-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:42:54 EDT 2025 oleg253-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg253-client.virtnet: executing check_config_client /mnt/lustre oleg253-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg253-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5cae000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5cae000.idle_timeout=debug disable quota as required oleg253-server: oleg253-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:04 (1748036584) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:06 EDT 2025 waiting for dbench pid 8762 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 517 35.58 MB/sec warmup 1 sec latency 13.557 ms 2 1163 23.01 MB/sec warmup 2 sec latency 12.532 ms 2 1805 17.55 MB/sec warmup 3 sec latency 12.762 ms 2 2447 14.06 MB/sec warmup 4 sec latency 10.535 ms 2 3543 15.35 MB/sec warmup 5 sec latency 11.065 ms 2 4066 13.36 MB/sec warmup 6 sec latency 9.662 ms 2 4691 12.03 MB/sec warmup 7 sec latency 11.921 ms 2 5302 11.35 MB/sec warmup 8 sec latency 10.556 ms 2 6004 10.49 MB/sec warmup 9 sec latency 10.920 ms 2 7099 11.51 MB/sec warmup 10 sec latency 11.112 ms 2 7589 10.76 MB/sec warmup 11 sec latency 11.944 ms 2 8808 6.59 MB/sec execute 1 sec latency 13.257 ms 2 9493 5.03 MB/sec execute 2 sec latency 10.702 ms 2 10450 8.30 MB/sec execute 3 sec latency 10.540 ms 2 11034 8.57 MB/sec execute 4 sec latency 10.645 ms 2 11551 7.15 MB/sec execute 5 sec latency 13.382 ms 2 12179 7.43 MB/sec execute 6 sec latency 11.022 ms 2 12793 6.53 MB/sec execute 7 sec latency 11.660 ms 2 13581 7.42 MB/sec execute 8 sec latency 11.565 ms 2 14425 7.99 MB/sec execute 9 sec latency 11.065 ms 2 14917 7.30 MB/sec execute 10 sec latency 13.074 ms 2 15445 6.98 MB/sec execute 11 sec latency 12.012 ms 2 16059 6.94 MB/sec execute 12 sec latency 11.094 ms 2 16678 6.77 MB/sec execute 13 sec latency 15.999 ms 2 17719 7.67 MB/sec execute 14 sec latency 11.160 ms 2 18182 7.38 MB/sec execute 15 sec latency 10.889 ms 2 18692 7.03 MB/sec execute 16 sec latency 14.701 ms 2 19318 7.12 MB/sec execute 17 sec latency 10.747 ms 2 19931 6.80 MB/sec execute 18 sec latency 10.863 ms 2 20682 7.08 MB/sec execute 19 sec latency 11.164 ms 2 21517 7.37 MB/sec execute 20 sec latency 10.583 ms 2 21991 7.11 MB/sec execute 21 sec latency 11.811 ms 2 22477 6.96 MB/sec execute 22 sec latency 20.811 ms 2 23057 6.94 MB/sec execute 23 sec latency 12.848 ms 2 23726 6.80 MB/sec execute 24 sec latency 9.088 ms 2 24666 7.12 MB/sec execute 25 sec latency 10.678 ms 2 25241 7.21 MB/sec execute 26 sec latency 10.611 ms 2 25749 6.99 MB/sec execute 27 sec latency 12.718 ms 2 26350 7.05 MB/sec execute 28 sec latency 10.633 ms 2 26897 6.85 MB/sec execute 29 sec latency 10.842 ms 2 27647 6.98 MB/sec execute 30 sec latency 10.108 ms 2 28549 7.19 MB/sec execute 31 sec latency 10.593 ms 2 28984 7.05 MB/sec execute 32 sec latency 11.563 ms 2 29527 6.96 MB/sec execute 33 sec latency 13.543 ms 2 30136 6.95 MB/sec execute 34 sec latency 12.266 ms 2 30797 6.85 MB/sec execute 35 sec latency 12.004 ms 2 31720 7.07 MB/sec execute 36 sec latency 11.113 ms 2 32313 7.13 MB/sec execute 37 sec latency 10.593 ms 2 32787 6.97 MB/sec execute 38 sec latency 14.370 ms 2 33403 7.02 MB/sec execute 39 sec latency 11.361 ms 2 33970 6.87 MB/sec execute 40 sec latency 17.232 ms 2 34573 6.83 MB/sec execute 41 sec latency 13.374 ms 2 35598 7.12 MB/sec execute 42 sec latency 10.649 ms 2 36021 7.02 MB/sec execute 43 sec latency 11.470 ms 2 36568 6.95 MB/sec execute 44 sec latency 16.130 ms 2 37170 6.95 MB/sec execute 45 sec latency 11.549 ms 2 37839 6.87 MB/sec execute 46 sec latency 11.161 ms 2 38673 6.99 MB/sec execute 47 sec latency 12.613 ms 2 39347 7.08 MB/sec execute 48 sec latency 10.922 ms 2 39840 6.96 MB/sec execute 49 sec latency 13.473 ms 2 40450 7.01 MB/sec execute 50 sec latency 10.757 ms 2 40987 6.89 MB/sec execute 51 sec latency 11.578 ms 2 41638 6.85 MB/sec execute 52 sec latency 11.207 ms 2 42636 7.08 MB/sec execute 53 sec latency 10.785 ms 2 43076 7.01 MB/sec execute 54 sec latency 11.063 ms 2 43597 6.92 MB/sec execute 55 sec latency 15.674 ms 2 44249 6.95 MB/sec execute 56 sec latency 13.512 ms 2 44944 6.88 MB/sec execute 57 sec latency 10.532 ms 2 45869 7.02 MB/sec execute 58 sec latency 10.166 ms 2 46508 7.06 MB/sec execute 59 sec latency 11.142 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13448 4.988 20.788 Close 9866 1.058 4.408 Rename 568 7.314 17.341 Unlink 2718 1.794 3.349 Qpathinfo 12204 1.215 11.650 Qfileinfo 2126 0.119 1.568 Qfsinfo 2228 0.080 1.471 Sfileinfo 1084 3.424 13.676 Find 4706 0.349 7.232 WriteX 6607 0.936 7.614 ReadX 21116 0.012 0.776 LockX 44 1.053 1.407 UnlockX 44 1.079 1.777 Flush 938 6.163 16.118 Throughput 7.06342 MB/sec 2 clients 2 procs max_latency=20.811 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:18 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 7 seconds Changelog records consumed: 19164 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (83s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:28 (1748036668) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:29 EDT 2025 waiting for dbench pid 9643 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 664 42.31 MB/sec warmup 1 sec latency 12.638 ms 2 1394 26.01 MB/sec warmup 2 sec latency 16.068 ms 2 2080 17.76 MB/sec warmup 3 sec latency 11.153 ms 2 2976 16.75 MB/sec warmup 4 sec latency 11.456 ms 2 3837 15.90 MB/sec warmup 5 sec latency 10.360 ms 2 4350 13.49 MB/sec warmup 6 sec latency 14.608 ms 2 5007 12.90 MB/sec warmup 7 sec latency 13.509 ms 2 5594 11.42 MB/sec warmup 8 sec latency 10.475 ms 2 6453 11.54 MB/sec warmup 9 sec latency 12.093 ms 2 7349 11.78 MB/sec warmup 10 sec latency 10.975 ms 2 7851 10.83 MB/sec warmup 11 sec latency 12.354 ms 2 9204 1.28 MB/sec execute 1 sec latency 16.622 ms 2 10076 7.49 MB/sec execute 2 sec latency 9.494 ms PIDs: 9643 9644 9645 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 4156 Resuming dbench 2 10647 4.26 MB/sec execute 6 sec latency 3339.914 ms 2 11154 4.12 MB/sec execute 7 sec latency 3339.928 ms 2 11792 4.11 MB/sec execute 8 sec latency 15.335 ms 2 12436 4.39 MB/sec execute 9 sec latency 13.735 ms 2 13122 4.42 MB/sec execute 10 sec latency 9.737 ms 2 14196 5.82 MB/sec execute 11 sec latency 10.511 ms 2 14667 5.61 MB/sec execute 12 sec latency 11.001 ms 2 15205 5.34 MB/sec execute 13 sec latency 13.540 ms 2 15863 5.56 MB/sec execute 14 sec latency 10.047 ms 2 16584 5.42 MB/sec execute 15 sec latency 11.178 ms 2 17551 6.02 MB/sec execute 16 sec latency 10.254 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 23320 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 2805 Wait for dbench to finish 2 17552 5.36 MB/sec execute 18 sec latency 1942.277 ms 2 18174 5.57 MB/sec execute 19 sec latency 1942.311 ms 2 18689 5.37 MB/sec execute 20 sec latency 32.487 ms 2 19347 5.54 MB/sec execute 21 sec latency 12.453 ms 2 19972 5.34 MB/sec execute 22 sec latency 10.243 ms 2 20804 5.70 MB/sec execute 23 sec latency 10.042 ms 2 21640 5.99 MB/sec execute 24 sec latency 12.244 ms 2 22198 5.82 MB/sec execute 25 sec latency 12.142 ms 2 22843 5.94 MB/sec execute 26 sec latency 13.226 ms 2 23491 5.77 MB/sec execute 27 sec latency 9.622 ms 2 24335 6.05 MB/sec execute 28 sec latency 9.869 ms 2 25180 6.28 MB/sec execute 29 sec latency 9.692 ms 2 25767 6.13 MB/sec execute 30 sec latency 12.752 ms 2 26484 6.23 MB/sec execute 31 sec latency 13.143 ms 2 27224 6.14 MB/sec execute 32 sec latency 10.265 ms 2 28222 6.41 MB/sec execute 33 sec latency 9.566 ms 2 28838 6.50 MB/sec execute 34 sec latency 10.296 ms 2 29406 6.36 MB/sec execute 35 sec latency 12.060 ms 2 30112 6.43 MB/sec execute 36 sec latency 9.802 ms 2 30827 6.35 MB/sec execute 37 sec latency 10.751 ms 2 31923 6.73 MB/sec execute 38 sec latency 9.257 ms 2 32434 6.64 MB/sec execute 39 sec latency 9.832 ms 2 33075 6.58 MB/sec execute 40 sec latency 16.464 ms 2 33753 6.58 MB/sec execute 41 sec latency 10.227 ms 2 34432 6.53 MB/sec execute 42 sec latency 9.686 ms 2 35558 6.84 MB/sec execute 43 sec latency 9.340 ms 2 36065 6.76 MB/sec execute 44 sec latency 11.101 ms 2 36694 6.70 MB/sec execute 45 sec latency 14.528 ms 2 37350 6.70 MB/sec execute 46 sec latency 14.282 ms 2 38090 6.67 MB/sec execute 47 sec latency 10.608 ms 2 39165 6.94 MB/sec execute 48 sec latency 14.666 ms 2 39724 6.86 MB/sec execute 49 sec latency 13.507 ms 2 40306 6.80 MB/sec execute 50 sec latency 19.643 ms 2 40959 6.79 MB/sec execute 51 sec latency 11.005 ms 2 41690 6.78 MB/sec execute 52 sec latency 9.565 ms 2 42772 7.01 MB/sec execute 53 sec latency 9.483 ms 2 43301 6.93 MB/sec execute 54 sec latency 15.146 ms 2 43988 6.98 MB/sec execute 55 sec latency 9.562 ms 2 44615 6.88 MB/sec execute 56 sec latency 13.527 ms 2 45493 6.98 MB/sec execute 57 sec latency 10.033 ms 2 46408 7.10 MB/sec execute 58 sec latency 13.730 ms 2 47003 7.01 MB/sec execute 59 sec latency 12.899 ms 2 47701 7.04 MB/sec execute 60 sec latency 10.439 ms 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13902 5.221 3339.919 Close 10256 0.940 5.787 Rename 574 6.587 10.935 Unlink 2754 1.656 3.955 Qpathinfo 12644 1.095 8.511 Qfileinfo 2198 0.112 1.340 Qfsinfo 2252 0.075 2.146 Sfileinfo 1102 3.087 16.450 Find 4830 0.312 6.741 WriteX 6824 0.853 4.611 ReadX 21496 0.011 0.768 LockX 44 0.922 1.354 UnlockX 44 0.931 1.125 Flush 958 5.422 19.636 Throughput 7.04151 MB/sec 2 clients 2 procs max_latency=3339.928 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:42 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 26124 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 12811 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (81s) == lustre-rsync-test test complete, duration 174 sec ===== 17:45:48 (1748036748)