-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:42:50 EDT 2025 oleg257-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg257-client.virtnet: executing check_config_client /mnt/lustre oleg257-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg257-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b58bb800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58bb800.idle_timeout=debug disable quota as required oleg257-server: oleg257-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:01 (1748036581) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:03 EDT 2025 waiting for dbench pid 8764 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 612 41.95 MB/sec warmup 1 sec latency 10.575 ms 2 1376 26.00 MB/sec warmup 2 sec latency 9.100 ms 2 2139 17.78 MB/sec warmup 3 sec latency 10.223 ms 2 3209 17.14 MB/sec warmup 4 sec latency 9.395 ms 2 3956 16.02 MB/sec warmup 5 sec latency 11.366 ms 2 4491 13.63 MB/sec warmup 6 sec latency 14.657 ms 2 5137 12.93 MB/sec warmup 7 sec latency 17.594 ms 2 5767 11.47 MB/sec warmup 8 sec latency 10.523 ms 2 6762 11.87 MB/sec warmup 9 sec latency 10.099 ms 2 7488 11.84 MB/sec warmup 10 sec latency 9.606 ms 2 8025 10.91 MB/sec warmup 11 sec latency 13.187 ms 2 9378 2.10 MB/sec execute 1 sec latency 10.680 ms 2 10311 8.13 MB/sec execute 2 sec latency 10.600 ms 2 11018 9.27 MB/sec execute 3 sec latency 9.549 ms 2 11547 7.32 MB/sec execute 4 sec latency 13.346 ms 2 12156 7.61 MB/sec execute 5 sec latency 11.139 ms 2 12767 6.55 MB/sec execute 6 sec latency 10.295 ms 2 13567 7.55 MB/sec execute 7 sec latency 10.506 ms 2 14424 8.18 MB/sec execute 8 sec latency 11.022 ms 2 14904 7.40 MB/sec execute 9 sec latency 11.899 ms 2 15433 7.03 MB/sec execute 10 sec latency 10.467 ms 2 16022 6.99 MB/sec execute 11 sec latency 11.039 ms 2 16663 6.78 MB/sec execute 12 sec latency 10.106 ms 2 17722 7.76 MB/sec execute 13 sec latency 9.916 ms 2 18185 7.45 MB/sec execute 14 sec latency 11.200 ms 2 18724 7.07 MB/sec execute 15 sec latency 12.030 ms 2 19348 7.17 MB/sec execute 16 sec latency 11.706 ms 2 19938 6.82 MB/sec execute 17 sec latency 11.643 ms 2 20705 7.19 MB/sec execute 18 sec latency 9.928 ms 2 21544 7.48 MB/sec execute 19 sec latency 12.103 ms 2 22006 7.15 MB/sec execute 20 sec latency 13.692 ms 2 22523 6.99 MB/sec execute 21 sec latency 10.556 ms 2 23100 6.97 MB/sec execute 22 sec latency 19.107 ms 2 23715 6.81 MB/sec execute 23 sec latency 12.330 ms 2 24620 7.14 MB/sec execute 24 sec latency 11.055 ms 2 25216 7.23 MB/sec execute 25 sec latency 11.924 ms 2 25689 7.00 MB/sec execute 26 sec latency 12.530 ms 2 26287 7.07 MB/sec execute 27 sec latency 11.705 ms 2 26859 6.85 MB/sec execute 28 sec latency 11.310 ms 2 27451 6.79 MB/sec execute 29 sec latency 11.879 ms 2 28452 7.20 MB/sec execute 30 sec latency 10.360 ms 2 28888 7.07 MB/sec execute 31 sec latency 10.716 ms 2 29400 6.90 MB/sec execute 32 sec latency 16.398 ms 2 30014 6.96 MB/sec execute 33 sec latency 11.984 ms 2 30626 6.79 MB/sec execute 34 sec latency 10.323 ms 2 31399 6.98 MB/sec execute 35 sec latency 10.464 ms 2 32207 7.13 MB/sec execute 36 sec latency 11.193 ms 2 32675 6.97 MB/sec execute 37 sec latency 12.982 ms 2 33186 6.88 MB/sec execute 38 sec latency 11.430 ms 2 33791 6.88 MB/sec execute 39 sec latency 11.216 ms 2 34424 6.80 MB/sec execute 40 sec latency 11.308 ms 2 35371 7.00 MB/sec execute 41 sec latency 10.419 ms 2 35915 7.04 MB/sec execute 42 sec latency 11.453 ms 2 36422 6.91 MB/sec execute 43 sec latency 15.292 ms 2 37020 6.95 MB/sec execute 44 sec latency 11.778 ms 2 37606 6.82 MB/sec execute 45 sec latency 11.024 ms 2 38378 6.92 MB/sec execute 46 sec latency 10.762 ms 2 39188 7.05 MB/sec execute 47 sec latency 11.589 ms 2 39619 6.96 MB/sec execute 48 sec latency 11.614 ms 2 40155 6.89 MB/sec execute 49 sec latency 14.600 ms 2 40752 6.89 MB/sec execute 50 sec latency 11.433 ms 2 41415 6.82 MB/sec execute 51 sec latency 7.856 ms 2 42263 6.93 MB/sec execute 52 sec latency 9.850 ms 2 42916 7.01 MB/sec execute 53 sec latency 11.523 ms 2 43387 6.91 MB/sec execute 54 sec latency 14.634 ms 2 43998 6.95 MB/sec execute 55 sec latency 10.497 ms 2 44512 6.84 MB/sec execute 56 sec latency 10.687 ms 2 45160 6.81 MB/sec execute 57 sec latency 12.937 ms 2 46157 7.02 MB/sec execute 58 sec latency 11.157 ms 2 46609 6.95 MB/sec execute 59 sec latency 11.524 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13310 5.034 18.035 Close 9774 1.061 5.103 Rename 560 7.283 11.963 Unlink 2704 1.821 3.266 Qpathinfo 12044 1.222 8.195 Qfileinfo 2102 0.129 1.922 Qfsinfo 2212 0.086 1.412 Sfileinfo 1068 3.531 11.176 Find 4664 0.364 7.465 WriteX 6566 0.961 8.258 ReadX 20854 0.013 0.826 LockX 44 0.966 1.234 UnlockX 44 1.097 2.094 Flush 944 6.072 19.093 Throughput 6.95405 MB/sec 2 clients 2 procs max_latency=19.107 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:15 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 7 seconds Changelog records consumed: 19224 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (84s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:25 (1748036665) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:27 EDT 2025 waiting for dbench pid 9646 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 674 42.35 MB/sec warmup 1 sec latency 17.411 ms 2 1512 26.07 MB/sec warmup 2 sec latency 7.838 ms 2 2416 18.73 MB/sec warmup 3 sec latency 7.923 ms 2 3690 19.37 MB/sec warmup 4 sec latency 8.617 ms 2 4281 16.13 MB/sec warmup 5 sec latency 12.923 ms 2 5012 15.04 MB/sec warmup 6 sec latency 11.299 ms 2 5707 13.09 MB/sec warmup 7 sec latency 8.688 ms 2 6552 13.16 MB/sec warmup 8 sec latency 10.154 ms 2 7429 13.08 MB/sec warmup 9 sec latency 9.920 ms 2 7986 11.98 MB/sec warmup 10 sec latency 11.687 ms 2 8660 11.70 MB/sec warmup 11 sec latency 12.151 ms 2 10308 14.99 MB/sec execute 1 sec latency 10.009 ms 2 11038 13.28 MB/sec execute 2 sec latency 9.268 ms PIDs: 9646 9647 9648 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 4586 Resuming dbench 2 11355 4.84 MB/sec execute 6 sec latency 3089.021 ms 2 12049 5.57 MB/sec execute 7 sec latency 3089.183 ms 2 12767 5.01 MB/sec execute 8 sec latency 9.223 ms 2 13680 6.04 MB/sec execute 9 sec latency 8.809 ms 2 14578 6.76 MB/sec execute 10 sec latency 9.368 ms 2 15157 6.28 MB/sec execute 11 sec latency 11.951 ms 2 15854 6.50 MB/sec execute 12 sec latency 13.897 ms 2 16585 6.26 MB/sec execute 13 sec latency 9.815 ms 2 17558 6.90 MB/sec execute 14 sec latency 10.139 ms 2 18174 7.06 MB/sec execute 15 sec latency 10.451 ms 2 18725 6.72 MB/sec execute 16 sec latency 12.100 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 23810 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 3003 Wait for dbench to finish 2 18727 6.05 MB/sec execute 17 sec latency 1719.543 ms 2 19436 6.20 MB/sec execute 18 sec latency 1719.568 ms 2 20162 6.07 MB/sec execute 19 sec latency 9.098 ms 2 21177 6.51 MB/sec execute 20 sec latency 34.924 ms 2 21832 6.63 MB/sec execute 21 sec latency 9.420 ms 2 22439 6.52 MB/sec execute 22 sec latency 12.110 ms 2 23117 6.52 MB/sec execute 23 sec latency 9.544 ms 2 23792 6.45 MB/sec execute 24 sec latency 11.604 ms 2 24943 6.98 MB/sec execute 25 sec latency 9.264 ms 2 25460 6.83 MB/sec execute 26 sec latency 10.272 ms 2 26095 6.73 MB/sec execute 27 sec latency 11.180 ms 2 26790 6.73 MB/sec execute 28 sec latency 9.488 ms 2 27637 6.88 MB/sec execute 29 sec latency 10.214 ms 2 28641 7.17 MB/sec execute 30 sec latency 9.437 ms 2 29171 6.98 MB/sec execute 31 sec latency 15.868 ms 2 29905 7.06 MB/sec execute 32 sec latency 8.848 ms 2 30629 6.89 MB/sec execute 33 sec latency 14.364 ms 2 31686 7.19 MB/sec execute 34 sec latency 10.692 ms 2 32367 7.25 MB/sec execute 35 sec latency 9.786 ms 2 32959 7.10 MB/sec execute 36 sec latency 14.612 ms 2 33689 7.15 MB/sec execute 37 sec latency 9.439 ms 2 34432 7.08 MB/sec execute 38 sec latency 7.702 ms 2 35585 7.41 MB/sec execute 39 sec latency 9.687 ms 2 36132 7.30 MB/sec execute 40 sec latency 9.439 ms 2 36740 7.23 MB/sec execute 41 sec latency 12.434 ms 2 37510 7.22 MB/sec execute 42 sec latency 9.357 ms 2 38422 7.37 MB/sec execute 43 sec latency 8.490 ms 2 39326 7.49 MB/sec execute 44 sec latency 10.796 ms 2 39917 7.37 MB/sec execute 45 sec latency 13.017 ms 2 40634 7.40 MB/sec execute 46 sec latency 16.911 ms 2 41404 7.32 MB/sec execute 47 sec latency 9.404 ms 2 42425 7.48 MB/sec execute 48 sec latency 8.827 ms 2 43064 7.52 MB/sec execute 49 sec latency 9.264 ms 2 43705 7.45 MB/sec execute 50 sec latency 13.289 ms 2 44435 7.44 MB/sec execute 51 sec latency 9.367 ms 2 45219 7.40 MB/sec execute 52 sec latency 8.324 ms 2 46326 7.64 MB/sec execute 53 sec latency 9.803 ms 2 46887 7.54 MB/sec execute 54 sec latency 11.402 ms 2 47605 7.58 MB/sec execute 55 sec latency 8.919 ms 2 48288 7.47 MB/sec execute 56 sec latency 13.593 ms 2 49143 7.58 MB/sec execute 57 sec latency 11.178 ms 2 50017 7.66 MB/sec execute 58 sec latency 10.246 ms 2 50596 7.57 MB/sec execute 59 sec latency 15.021 ms 2 cleanup 60 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 14474 4.693 3089.176 Close 10570 0.887 7.776 Rename 622 6.211 10.128 Unlink 2960 1.542 3.208 Qpathinfo 13066 1.061 18.822 Qfileinfo 2284 0.113 1.178 Qfsinfo 2448 0.076 1.299 Sfileinfo 1200 2.920 15.014 Find 5094 0.321 6.395 WriteX 7182 1.306 1719.559 ReadX 22926 0.011 0.281 LockX 48 0.892 1.167 UnlockX 48 0.876 1.080 Flush 1024 5.124 34.910 Throughput 7.56543 MB/sec 2 clients 2 procs max_latency=3089.183 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:39 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 26812 Clear changelog after use: no Errors: 0 lustre_rsync took 5 seconds Changelog records consumed: 13333 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (81s) == lustre-rsync-test test complete, duration 176 sec ===== 17:45:47 (1748036747)