-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:43:05 EDT 2025 oleg126-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg126-client.virtnet: executing check_config_client /mnt/lustre oleg126-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg126-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800ad949800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800ad949800.idle_timeout=debug disable quota as required oleg126-server: oleg126-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:16 (1748036596) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:18 EDT 2025 waiting for dbench pid 9607 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 490 34.15 MB/sec warmup 1 sec latency 17.243 ms 2 1136 22.99 MB/sec warmup 2 sec latency 10.315 ms 2 1810 17.56 MB/sec warmup 3 sec latency 12.210 ms 2 2558 14.45 MB/sec warmup 4 sec latency 10.248 ms 2 3636 15.45 MB/sec warmup 5 sec latency 11.576 ms 2 4152 13.36 MB/sec warmup 6 sec latency 10.747 ms 2 4746 12.04 MB/sec warmup 7 sec latency 13.642 ms 2 5397 11.37 MB/sec warmup 8 sec latency 9.559 ms 2 6125 10.67 MB/sec warmup 9 sec latency 10.255 ms 2 7226 11.57 MB/sec warmup 10 sec latency 9.571 ms 2 7751 10.78 MB/sec warmup 11 sec latency 10.081 ms 2 8953 6.61 MB/sec execute 1 sec latency 26.832 ms 2 9679 5.90 MB/sec execute 2 sec latency 11.134 ms 2 10765 10.45 MB/sec execute 3 sec latency 10.116 ms 2 11297 8.56 MB/sec execute 4 sec latency 11.344 ms 2 11881 7.66 MB/sec execute 5 sec latency 15.240 ms 2 12582 7.51 MB/sec execute 6 sec latency 11.466 ms 2 13285 7.28 MB/sec execute 7 sec latency 10.222 ms 2 14347 8.71 MB/sec execute 8 sec latency 10.009 ms 2 14895 8.08 MB/sec execute 9 sec latency 24.403 ms 2 15454 7.66 MB/sec execute 10 sec latency 11.879 ms 2 16130 7.57 MB/sec execute 11 sec latency 10.006 ms 2 16822 7.40 MB/sec execute 12 sec latency 10.624 ms 2 17879 8.30 MB/sec execute 13 sec latency 11.176 ms 2 18429 7.93 MB/sec execute 14 sec latency 12.197 ms 2 19022 7.66 MB/sec execute 15 sec latency 15.663 ms 2 19731 7.60 MB/sec execute 16 sec latency 10.003 ms 2 20530 7.80 MB/sec execute 17 sec latency 10.353 ms 2 21486 8.13 MB/sec execute 18 sec latency 10.559 ms 2 22028 7.86 MB/sec execute 19 sec latency 14.845 ms 2 22709 7.95 MB/sec execute 20 sec latency 10.338 ms 2 23295 7.61 MB/sec execute 21 sec latency 60.238 ms 2 24094 7.77 MB/sec execute 22 sec latency 11.012 ms 2 25070 8.11 MB/sec execute 23 sec latency 9.993 ms 2 25583 7.82 MB/sec execute 24 sec latency 13.856 ms 2 26158 7.66 MB/sec execute 25 sec latency 9.508 ms 2 26881 7.62 MB/sec execute 26 sec latency 15.046 ms 2 27643 7.75 MB/sec execute 27 sec latency 11.364 ms 2 28619 8.03 MB/sec execute 28 sec latency 9.303 ms 2 29123 7.79 MB/sec execute 29 sec latency 12.375 ms 2 29686 7.66 MB/sec execute 30 sec latency 14.422 ms 2 30385 7.62 MB/sec execute 31 sec latency 12.158 ms 2 31183 7.73 MB/sec execute 32 sec latency 10.287 ms 2 32131 7.91 MB/sec execute 33 sec latency 10.308 ms 2 32664 7.77 MB/sec execute 34 sec latency 14.343 ms 2 33239 7.66 MB/sec execute 35 sec latency 13.668 ms 2 33942 7.63 MB/sec execute 36 sec latency 10.758 ms 2 34738 7.72 MB/sec execute 37 sec latency 9.239 ms 2 35725 7.93 MB/sec execute 38 sec latency 9.454 ms 2 36231 7.75 MB/sec execute 39 sec latency 14.760 ms 2 36910 7.80 MB/sec execute 40 sec latency 9.452 ms 2 37530 7.63 MB/sec execute 41 sec latency 16.746 ms 2 38401 7.75 MB/sec execute 42 sec latency 10.156 ms 2 39310 7.89 MB/sec execute 43 sec latency 9.470 ms 2 39861 7.75 MB/sec execute 44 sec latency 12.810 ms 2 40562 7.78 MB/sec execute 45 sec latency 10.244 ms 2 41299 7.65 MB/sec execute 46 sec latency 9.096 ms 2 42298 7.82 MB/sec execute 47 sec latency 10.373 ms 2 43009 7.88 MB/sec execute 48 sec latency 9.975 ms 2 43656 7.80 MB/sec execute 49 sec latency 14.562 ms 2 44379 7.78 MB/sec execute 50 sec latency 16.346 ms 2 45193 7.73 MB/sec execute 51 sec latency 9.931 ms 2 46380 8.00 MB/sec execute 52 sec latency 8.828 ms 2 46979 7.88 MB/sec execute 53 sec latency 12.746 ms 2 47735 7.91 MB/sec execute 54 sec latency 10.672 ms 2 48524 7.83 MB/sec execute 55 sec latency 9.684 ms 2 49648 8.06 MB/sec execute 56 sec latency 9.779 ms 2 50193 7.98 MB/sec execute 57 sec latency 9.639 ms 2 50877 7.91 MB/sec execute 58 sec latency 14.090 ms 2 51651 7.89 MB/sec execute 59 sec latency 10.985 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 15310 4.346 17.327 Close 11296 0.900 2.514 Rename 646 6.466 10.546 Unlink 3036 1.583 3.602 Qpathinfo 13924 1.099 9.347 Qfileinfo 2452 0.120 1.592 Qfsinfo 2502 0.076 1.459 Sfileinfo 1246 2.970 15.648 Find 5344 0.329 6.788 WriteX 7558 0.858 8.542 ReadX 24098 0.011 0.156 LockX 52 0.875 1.148 UnlockX 52 0.901 1.142 Flush 1054 5.404 60.230 Throughput 7.89418 MB/sec 2 clients 2 procs max_latency=60.238 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:30 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 8 seconds Changelog records consumed: 21386 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0001: changelog user 'cl1' not found lustre-MDT0000: changelog user 'cl1' not found PASS 2a (86s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:42 (1748036682) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:45 EDT 2025 waiting for dbench pid 10579 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 468 32.85 MB/sec warmup 1 sec latency 12.953 ms 2 1080 22.93 MB/sec warmup 2 sec latency 10.917 ms 2 1789 17.54 MB/sec warmup 3 sec latency 10.971 ms 2 2594 14.50 MB/sec warmup 4 sec latency 8.986 ms 2 3768 15.90 MB/sec warmup 5 sec latency 9.553 ms 2 4301 13.48 MB/sec warmup 6 sec latency 13.576 ms 2 5023 12.90 MB/sec warmup 7 sec latency 10.130 ms 2 5650 11.44 MB/sec warmup 8 sec latency 17.244 ms 2 6489 11.69 MB/sec warmup 9 sec latency 11.679 ms 2 7356 11.78 MB/sec warmup 10 sec latency 10.834 ms 2 7884 10.84 MB/sec warmup 11 sec latency 16.840 ms 2 9127 1.04 MB/sec execute 1 sec latency 10.494 ms 2 9979 6.18 MB/sec execute 2 sec latency 10.289 ms PIDs: 10579 10580 10581 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 4074 Resuming dbench 2 10439 3.30 MB/sec execute 6 sec latency 3258.680 ms 2 11057 4.21 MB/sec execute 7 sec latency 12.293 ms 2 11609 3.88 MB/sec execute 8 sec latency 13.645 ms 2 12246 4.44 MB/sec execute 9 sec latency 14.555 ms 2 12905 4.11 MB/sec execute 10 sec latency 10.486 ms 2 13826 5.10 MB/sec execute 11 sec latency 11.081 ms 2 14524 5.63 MB/sec execute 12 sec latency 10.975 ms 2 15069 5.33 MB/sec execute 13 sec latency 13.570 ms 2 15702 5.60 MB/sec execute 14 sec latency 14.584 ms 2 16348 5.30 MB/sec execute 15 sec latency 9.826 ms 2 17126 5.83 MB/sec execute 16 sec latency 15.991 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 25460 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 2801 Wait for dbench to finish 2 17126 5.17 MB/sec execute 18 sec latency 1993.679 ms 2 17998 5.57 MB/sec execute 19 sec latency 1993.748 ms 2 18533 5.36 MB/sec execute 20 sec latency 13.105 ms 2 19177 5.55 MB/sec execute 21 sec latency 12.165 ms 2 19808 5.35 MB/sec execute 22 sec latency 9.774 ms 2 20501 5.59 MB/sec execute 23 sec latency 25.743 ms 2 21466 5.93 MB/sec execute 24 sec latency 12.976 ms 2 21991 5.80 MB/sec execute 25 sec latency 10.697 ms 2 22557 5.73 MB/sec execute 26 sec latency 16.544 ms 2 23172 5.76 MB/sec execute 27 sec latency 10.157 ms 2 23899 5.74 MB/sec execute 28 sec latency 10.585 ms 2 24954 6.22 MB/sec execute 29 sec latency 10.836 ms 2 25450 6.11 MB/sec execute 30 sec latency 11.023 ms 2 26038 6.04 MB/sec execute 31 sec latency 13.221 ms 2 26674 6.06 MB/sec execute 32 sec latency 11.406 ms 2 27336 6.00 MB/sec execute 33 sec latency 10.602 ms 2 28422 6.42 MB/sec execute 34 sec latency 9.482 ms 2 28904 6.33 MB/sec execute 35 sec latency 9.935 ms 2 29542 6.26 MB/sec execute 36 sec latency 13.302 ms 2 30218 6.28 MB/sec execute 37 sec latency 13.615 ms 2 30908 6.24 MB/sec execute 38 sec latency 9.358 ms 2 32051 6.59 MB/sec execute 39 sec latency 9.830 ms 2 32617 6.50 MB/sec execute 40 sec latency 10.410 ms 2 33232 6.44 MB/sec execute 41 sec latency 15.258 ms 2 33990 6.45 MB/sec execute 42 sec latency 10.008 ms 2 34889 6.63 MB/sec execute 43 sec latency 10.753 ms 2 35795 6.76 MB/sec execute 44 sec latency 8.923 ms 2 36407 6.66 MB/sec execute 45 sec latency 21.815 ms 2 37139 6.71 MB/sec execute 46 sec latency 13.357 ms 2 37911 6.64 MB/sec execute 47 sec latency 9.729 ms 2 39065 6.94 MB/sec execute 48 sec latency 8.390 ms 2 39717 6.87 MB/sec execute 49 sec latency 8.547 ms 2 40481 6.93 MB/sec execute 50 sec latency 20.984 ms 2 41258 6.82 MB/sec execute 51 sec latency 13.301 ms 2 42427 7.03 MB/sec execute 52 sec latency 8.550 ms 2 43125 7.07 MB/sec execute 53 sec latency 8.608 ms 2 43861 7.02 MB/sec execute 54 sec latency 13.805 ms 2 44654 7.02 MB/sec execute 55 sec latency 11.330 ms 2 45616 7.14 MB/sec execute 56 sec latency 8.015 ms 2 46411 7.23 MB/sec execute 57 sec latency 17.627 ms 2 46858 7.13 MB/sec execute 58 sec latency 34.565 ms 2 47207 7.06 MB/sec execute 59 sec latency 18.107 ms 2 47706 7.05 MB/sec execute 60 sec latency 17.617 ms 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13749 4.495 25.719 Close 10094 0.940 6.144 Rename 574 6.570 17.574 Unlink 2742 1.645 6.633 Qpathinfo 12422 1.440 1993.735 Qfileinfo 2170 0.123 2.278 Qfsinfo 2244 0.080 2.457 Sfileinfo 1100 3.136 17.115 Find 4774 0.330 9.121 WriteX 6730 0.876 8.542 ReadX 21326 0.011 1.257 LockX 44 0.878 1.143 UnlockX 44 0.895 1.146 Flush 952 12.310 3258.673 Throughput 7.05389 MB/sec 2 clients 2 procs max_latency=3258.680 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:58 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 28260 Clear changelog after use: no Errors: 0 lustre_rsync took 11 seconds Changelog records consumed: 12745 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0001: changelog user 'cl2' not found lustre-MDT0000: changelog user 'cl2' not found PASS 2b (91s) == lustre-rsync-test test complete, duration 188 sec ===== 17:46:13 (1748036773)