-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:42:56 EDT 2025 oleg228-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg228-client.virtnet: executing check_config_client /mnt/lustre oleg228-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg228-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a66b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a66b800.idle_timeout=debug disable quota as required oleg228-server: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:06 (1748036586) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:08 EDT 2025 waiting for dbench pid 8776 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 473 33.11 MB/sec warmup 1 sec latency 16.536 ms 2 1067 22.90 MB/sec warmup 2 sec latency 11.819 ms 2 1751 17.52 MB/sec warmup 3 sec latency 11.237 ms 2 2461 14.20 MB/sec warmup 4 sec latency 11.188 ms 2 3626 15.44 MB/sec warmup 5 sec latency 9.501 ms 2 4117 13.36 MB/sec warmup 6 sec latency 10.436 ms 2 4747 12.04 MB/sec warmup 7 sec latency 12.688 ms 2 5395 11.37 MB/sec warmup 8 sec latency 10.696 ms 2 6141 10.68 MB/sec warmup 9 sec latency 9.927 ms 2 7229 11.57 MB/sec warmup 10 sec latency 10.331 ms 2 7726 10.77 MB/sec warmup 11 sec latency 10.629 ms 2 8908 6.64 MB/sec execute 1 sec latency 14.007 ms 2 9581 5.70 MB/sec execute 2 sec latency 11.218 ms 2 10660 10.27 MB/sec execute 3 sec latency 9.546 ms 2 11136 8.52 MB/sec execute 4 sec latency 10.315 ms 2 11721 7.60 MB/sec execute 5 sec latency 15.791 ms 2 12341 7.44 MB/sec execute 6 sec latency 11.766 ms 2 13007 6.86 MB/sec execute 7 sec latency 9.955 ms 2 13922 7.86 MB/sec execute 8 sec latency 10.347 ms 2 14549 8.04 MB/sec execute 9 sec latency 11.120 ms 2 15073 7.36 MB/sec execute 10 sec latency 13.178 ms 2 15670 7.50 MB/sec execute 11 sec latency 16.071 ms 2 16221 6.96 MB/sec execute 12 sec latency 13.102 ms 2 16966 7.26 MB/sec execute 13 sec latency 11.316 ms 2 17879 7.71 MB/sec execute 14 sec latency 10.697 ms 2 18363 7.37 MB/sec execute 15 sec latency 11.107 ms 2 18907 7.17 MB/sec execute 16 sec latency 14.144 ms 2 19537 7.13 MB/sec execute 17 sec latency 11.364 ms 2 20185 6.93 MB/sec execute 18 sec latency 11.818 ms 2 21140 7.35 MB/sec execute 19 sec latency 11.683 ms 2 21717 7.45 MB/sec execute 20 sec latency 11.635 ms 2 22238 7.16 MB/sec execute 21 sec latency 13.851 ms 2 22833 7.23 MB/sec execute 22 sec latency 16.258 ms 2 23465 6.97 MB/sec execute 23 sec latency 9.793 ms 2 24239 7.25 MB/sec execute 24 sec latency 11.498 ms 2 25095 7.46 MB/sec execute 25 sec latency 10.400 ms 2 25585 7.22 MB/sec execute 26 sec latency 13.054 ms 2 26107 7.09 MB/sec execute 27 sec latency 18.903 ms 2 26736 7.07 MB/sec execute 28 sec latency 10.572 ms 2 27445 7.01 MB/sec execute 29 sec latency 10.547 ms 2 28452 7.40 MB/sec execute 30 sec latency 13.055 ms 2 28918 7.27 MB/sec execute 31 sec latency 11.497 ms 2 29492 7.17 MB/sec execute 32 sec latency 15.278 ms 2 30118 7.15 MB/sec execute 33 sec latency 11.360 ms 2 30814 7.04 MB/sec execute 34 sec latency 9.935 ms 2 31776 7.27 MB/sec execute 35 sec latency 10.363 ms 2 32350 7.32 MB/sec execute 36 sec latency 10.691 ms 2 32867 7.16 MB/sec execute 37 sec latency 18.254 ms 2 33509 7.21 MB/sec execute 38 sec latency 10.525 ms 2 34115 7.05 MB/sec execute 39 sec latency 10.644 ms 2 34940 7.22 MB/sec execute 40 sec latency 11.107 ms 2 35767 7.35 MB/sec execute 41 sec latency 11.163 ms 2 36249 7.20 MB/sec execute 42 sec latency 15.826 ms 2 36828 7.16 MB/sec execute 43 sec latency 10.087 ms 2 37460 7.11 MB/sec execute 44 sec latency 10.223 ms 2 38111 7.07 MB/sec execute 45 sec latency 10.341 ms 2 39165 7.33 MB/sec execute 46 sec latency 9.178 ms 2 39658 7.24 MB/sec execute 47 sec latency 10.300 ms 2 40218 7.17 MB/sec execute 48 sec latency 17.020 ms 2 40845 7.16 MB/sec execute 49 sec latency 11.472 ms 2 41512 7.09 MB/sec execute 50 sec latency 9.600 ms 2 42572 7.35 MB/sec execute 51 sec latency 10.368 ms 2 43046 7.28 MB/sec execute 52 sec latency 10.876 ms 2 43588 7.17 MB/sec execute 53 sec latency 12.037 ms 2 44258 7.20 MB/sec execute 54 sec latency 9.524 ms 2 44966 7.13 MB/sec execute 55 sec latency 9.292 ms 2 45962 7.27 MB/sec execute 56 sec latency 9.708 ms 2 46581 7.31 MB/sec execute 57 sec latency 10.037 ms 2 47145 7.22 MB/sec execute 58 sec latency 13.162 ms 2 47859 7.24 MB/sec execute 59 sec latency 9.699 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 14018 4.791 15.716 Close 10348 0.993 2.670 Rename 586 6.912 11.653 Unlink 2776 1.716 3.121 Qpathinfo 12762 1.159 11.348 Qfileinfo 2232 0.126 1.511 Qfsinfo 2272 0.081 1.367 Sfileinfo 1138 3.435 18.888 Find 4882 0.345 7.146 WriteX 6910 0.920 5.926 ReadX 21720 0.013 0.840 LockX 44 0.976 1.428 UnlockX 44 1.010 1.430 Flush 964 5.765 17.007 Throughput 7.23943 MB/sec 2 clients 2 procs max_latency=18.903 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:21 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 6 seconds Changelog records consumed: 19850 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (83s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:29 (1748036669) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:31 EDT 2025 waiting for dbench pid 9660 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 522 36.11 MB/sec warmup 1 sec latency 11.619 ms 2 1185 23.03 MB/sec warmup 2 sec latency 21.660 ms 2 1888 17.59 MB/sec warmup 3 sec latency 10.328 ms 2 2586 14.48 MB/sec warmup 4 sec latency 10.225 ms 2 3678 15.49 MB/sec warmup 5 sec latency 10.920 ms 2 4180 13.36 MB/sec warmup 6 sec latency 10.745 ms 2 4744 12.04 MB/sec warmup 7 sec latency 14.080 ms 2 5385 11.36 MB/sec warmup 8 sec latency 10.326 ms 2 6116 10.67 MB/sec warmup 9 sec latency 11.346 ms 2 7181 11.55 MB/sec warmup 10 sec latency 9.810 ms 2 7695 10.76 MB/sec warmup 11 sec latency 10.241 ms 2 8986 6.64 MB/sec execute 1 sec latency 14.359 ms 2 9820 8.88 MB/sec execute 2 sec latency 9.185 ms PIDs: 9660 9661 9662 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 2 seconds Changelog records consumed: 4074 Resuming dbench 2 10403 4.33 MB/sec execute 6 sec latency 3194.792 ms 2 11060 5.08 MB/sec execute 7 sec latency 3194.810 ms 2 11625 4.65 MB/sec execute 8 sec latency 14.535 ms 2 12298 5.12 MB/sec execute 9 sec latency 16.118 ms 2 12954 4.92 MB/sec execute 10 sec latency 10.901 ms 2 13857 5.68 MB/sec execute 11 sec latency 10.348 ms 2 14523 6.13 MB/sec execute 12 sec latency 10.775 ms 2 15056 5.79 MB/sec execute 13 sec latency 12.719 ms 2 15707 6.02 MB/sec execute 14 sec latency 13.262 ms 2 16337 5.70 MB/sec execute 15 sec latency 10.600 ms 2 17171 6.21 MB/sec execute 16 sec latency 11.445 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 23924 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 2809 Wait for dbench to finish 2 17174 5.56 MB/sec execute 18 sec latency 1820.227 ms 2 18009 5.94 MB/sec execute 19 sec latency 1820.324 ms 2 18541 5.71 MB/sec execute 20 sec latency 14.744 ms 2 19069 5.70 MB/sec execute 21 sec latency 45.527 ms 2 19733 5.65 MB/sec execute 22 sec latency 11.474 ms 2 20416 5.66 MB/sec execute 23 sec latency 9.999 ms 2 21478 6.22 MB/sec execute 24 sec latency 9.758 ms 2 22014 6.09 MB/sec execute 25 sec latency 14.396 ms 2 22575 6.00 MB/sec execute 26 sec latency 12.968 ms 2 23222 6.02 MB/sec execute 27 sec latency 17.011 ms 2 23933 6.00 MB/sec execute 28 sec latency 8.587 ms 2 25031 6.47 MB/sec execute 29 sec latency 9.128 ms 2 25599 6.36 MB/sec execute 30 sec latency 14.023 ms 2 26300 6.46 MB/sec execute 31 sec latency 9.501 ms 2 26912 6.29 MB/sec execute 32 sec latency 12.507 ms 2 27789 6.52 MB/sec execute 33 sec latency 8.839 ms 2 28675 6.71 MB/sec execute 34 sec latency 10.211 ms 2 29217 6.56 MB/sec execute 35 sec latency 11.863 ms 2 29905 6.63 MB/sec execute 36 sec latency 10.378 ms 2 30579 6.49 MB/sec execute 37 sec latency 9.874 ms 2 31427 6.68 MB/sec execute 38 sec latency 9.621 ms 2 32296 6.83 MB/sec execute 39 sec latency 9.721 ms 2 32857 6.71 MB/sec execute 40 sec latency 13.023 ms 2 33534 6.76 MB/sec execute 41 sec latency 14.990 ms 2 34238 6.65 MB/sec execute 42 sec latency 9.536 ms 2 35232 6.88 MB/sec execute 43 sec latency 11.478 ms 2 35895 6.94 MB/sec execute 44 sec latency 10.092 ms 2 36452 6.82 MB/sec execute 45 sec latency 12.701 ms 2 37160 6.86 MB/sec execute 46 sec latency 11.116 ms 2 37876 6.79 MB/sec execute 47 sec latency 9.937 ms 2 38876 6.96 MB/sec execute 48 sec latency 9.853 ms 2 39501 7.01 MB/sec execute 49 sec latency 9.318 ms 2 40055 6.91 MB/sec execute 50 sec latency 13.711 ms 2 40749 6.94 MB/sec execute 51 sec latency 10.389 ms 2 41477 6.88 MB/sec execute 52 sec latency 10.062 ms 2 42567 7.14 MB/sec execute 53 sec latency 11.037 ms 2 43077 7.07 MB/sec execute 54 sec latency 9.721 ms 2 43705 7.01 MB/sec execute 55 sec latency 14.173 ms 2 44378 7.01 MB/sec execute 56 sec latency 9.661 ms 2 45081 6.97 MB/sec execute 57 sec latency 8.430 ms 2 46254 7.20 MB/sec execute 58 sec latency 8.564 ms 2 46816 7.12 MB/sec execute 59 sec latency 9.100 ms 2 47426 7.07 MB/sec execute 60 sec latency 11.995 ms 2 cleanup 61 sec 0 cleanup 61 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 13896 4.463 34.753 Close 10256 0.929 8.944 Rename 580 6.530 12.956 Unlink 2758 1.649 3.712 Qpathinfo 12626 1.089 8.162 Qfileinfo 2198 0.118 1.384 Qfsinfo 2254 0.076 1.102 Sfileinfo 1114 3.047 14.397 Find 4830 0.318 7.495 WriteX 6806 1.330 3194.553 ReadX 21602 0.012 0.302 LockX 44 0.874 1.375 UnlockX 44 0.888 1.204 Flush 958 12.678 3194.793 Throughput 7.07229 MB/sec 2 clients 2 procs max_latency=3194.810 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:44 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 26732 Clear changelog after use: no Errors: 0 lustre_rsync took 4 seconds Changelog records consumed: 12823 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (81s) == lustre-rsync-test test complete, duration 173 sec ===== 17:45:50 (1748036750)