-----============= acceptance-small: lustre-rsync-test ============----- Fri May 23 17:42:59 EDT 2025 oleg416-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/lustre-rsync-test.*ex: No such file or directory excepting tests: 2b oleg416-client.virtnet: executing check_config_client /mnt/lustre oleg416-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg416-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5b43000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5b43000.idle_timeout=debug disable quota as required oleg416-server: oleg416-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 osd-ldiskfs.track_declares_assert=1 nobody:x:99: == lustre-rsync-test test 2a: Replicate files created by dbench. ========================================================== 17:43:11 (1748036591) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000: Registered changelog user cl1 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2a.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:43:13 EDT 2025 waiting for dbench pid 8757 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs failed to create barrier semaphore 0 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 662 42.30 MB/sec warmup 1 sec latency 12.030 ms 2 1457 26.03 MB/sec warmup 2 sec latency 8.570 ms 2 2334 18.55 MB/sec warmup 3 sec latency 8.001 ms 2 3531 19.19 MB/sec warmup 4 sec latency 7.996 ms 2 4226 16.07 MB/sec warmup 5 sec latency 8.445 ms 2 5034 15.05 MB/sec warmup 6 sec latency 12.671 ms 2 5907 13.43 MB/sec warmup 7 sec latency 8.366 ms 2 7131 14.39 MB/sec warmup 8 sec latency 7.690 ms 2 7787 13.20 MB/sec warmup 9 sec latency 10.824 ms 2 8556 12.85 MB/sec warmup 10 sec latency 8.149 ms 2 9343 11.82 MB/sec warmup 11 sec latency 10.203 ms 2 11184 9.27 MB/sec execute 1 sec latency 9.991 ms 2 11930 6.81 MB/sec execute 2 sec latency 11.408 ms 2 12800 6.90 MB/sec execute 3 sec latency 8.677 ms 2 13896 9.22 MB/sec execute 4 sec latency 9.404 ms 2 14662 9.49 MB/sec execute 5 sec latency 8.311 ms 2 15392 8.61 MB/sec execute 6 sec latency 11.152 ms 2 16207 8.37 MB/sec execute 7 sec latency 8.693 ms 2 17188 9.08 MB/sec execute 8 sec latency 8.918 ms 2 18133 9.52 MB/sec execute 9 sec latency 8.165 ms 2 18803 8.96 MB/sec execute 10 sec latency 12.395 ms 2 19583 8.77 MB/sec execute 11 sec latency 8.164 ms 2 20416 8.56 MB/sec execute 12 sec latency 9.298 ms 2 21585 9.50 MB/sec execute 13 sec latency 8.714 ms 2 22213 8.96 MB/sec execute 14 sec latency 13.056 ms 2 22985 8.97 MB/sec execute 15 sec latency 9.276 ms 2 23775 8.65 MB/sec execute 16 sec latency 9.890 ms 2 24993 9.39 MB/sec execute 17 sec latency 9.192 ms 2 25629 9.06 MB/sec execute 18 sec latency 13.009 ms 2 26420 9.08 MB/sec execute 19 sec latency 11.117 ms 2 27243 8.81 MB/sec execute 20 sec latency 8.063 ms 2 28417 9.39 MB/sec execute 21 sec latency 8.292 ms 2 29010 9.12 MB/sec execute 22 sec latency 21.838 ms 2 29691 8.91 MB/sec execute 23 sec latency 13.732 ms 2 30504 8.83 MB/sec execute 24 sec latency 9.503 ms 2 31441 9.03 MB/sec execute 25 sec latency 13.098 ms 2 32375 9.18 MB/sec execute 26 sec latency 8.147 ms 2 33052 8.99 MB/sec execute 27 sec latency 13.591 ms 2 33818 8.91 MB/sec execute 28 sec latency 9.364 ms 2 34743 8.99 MB/sec execute 29 sec latency 9.553 ms 2 35787 9.21 MB/sec execute 30 sec latency 10.089 ms 2 36416 8.98 MB/sec execute 31 sec latency 12.777 ms 2 37185 8.99 MB/sec execute 32 sec latency 8.522 ms 2 37976 8.86 MB/sec execute 33 sec latency 8.213 ms 2 39173 9.19 MB/sec execute 34 sec latency 8.268 ms 2 39790 9.03 MB/sec execute 35 sec latency 11.692 ms 2 40565 9.04 MB/sec execute 36 sec latency 10.199 ms 2 41412 8.90 MB/sec execute 37 sec latency 9.191 ms 2 42590 9.21 MB/sec execute 38 sec latency 8.442 ms 2 43218 9.06 MB/sec execute 39 sec latency 8.513 ms 2 43998 9.09 MB/sec execute 40 sec latency 11.959 ms 2 44764 8.90 MB/sec execute 41 sec latency 12.007 ms 2 45935 9.10 MB/sec execute 42 sec latency 9.842 ms 2 46624 9.11 MB/sec execute 43 sec latency 8.195 ms 2 47335 9.00 MB/sec execute 44 sec latency 13.309 ms 2 48118 8.95 MB/sec execute 45 sec latency 13.754 ms 2 49116 9.06 MB/sec execute 46 sec latency 8.752 ms 2 50054 9.15 MB/sec execute 47 sec latency 8.385 ms 2 50683 9.00 MB/sec execute 48 sec latency 12.809 ms 2 51479 8.99 MB/sec execute 49 sec latency 10.950 ms 2 52300 8.92 MB/sec execute 50 sec latency 8.850 ms 2 53502 9.17 MB/sec execute 51 sec latency 7.650 ms 2 54139 9.03 MB/sec execute 52 sec latency 12.394 ms 2 54904 9.03 MB/sec execute 53 sec latency 12.444 ms 2 55728 8.96 MB/sec execute 54 sec latency 9.287 ms 2 56984 9.20 MB/sec execute 55 sec latency 7.991 ms 2 57608 9.06 MB/sec execute 56 sec latency 11.020 ms 2 58432 9.07 MB/sec execute 57 sec latency 12.514 ms 2 59249 9.00 MB/sec execute 58 sec latency 8.730 ms 2 60482 9.19 MB/sec execute 59 sec latency 7.726 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 17634 3.783 21.815 Close 12880 0.773 5.656 Rename 740 5.599 8.767 Unlink 3626 1.390 4.099 Qpathinfo 15920 0.958 6.211 Qfileinfo 2738 0.105 1.241 Qfsinfo 2938 0.062 0.963 Sfileinfo 1414 2.596 13.050 Find 6108 0.297 5.169 WriteX 8608 0.747 4.066 ReadX 27356 0.010 0.967 LockX 56 0.732 0.878 UnlockX 56 0.775 0.960 Flush 1228 4.591 13.724 Throughput 9.1878 MB/sec 2 clients 2 procs max_latency=21.838 ms stopping dbench in /mnt/lustre/d2a.lustre-rsync-test at Fri May 23 17:44:25 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test /mnt/lustre/d2a.lustre-rsync-test removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2a.lustre-rsync-test dbench successfully finished Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl1 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 8 seconds Changelog records consumed: 24858 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 lustre-MDT0000: changelog user 'cl1' not found PASS 2a (85s) == lustre-rsync-test test 2b: Replicate files changed by dbench. ========================================================== 17:44:36 (1748036676) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' lustre-MDT0000: Registered changelog user cl2 waiting child_pid=$(pgrep dbench), 360 secs left ... looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' copying required dbench files to chroot directory /mnt/lustre/d2b.lustre-rsync-test '/usr/bin/dbench' -> 'dbench' lib64/libpopt.so.0 lib64/libc.so.6 lib64/ld-linux-x86-64.so.2 running 'dbench 2 -t 60' in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:44:38 EDT 2025 waiting for dbench pid 9640 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 60 seconds with load 'client.txt' and minimum warmup 12 secs 1 of 2 processes prepared for launch 0 sec 2 of 2 processes prepared for launch 0 sec releasing clients 2 656 42.13 MB/sec warmup 1 sec latency 10.368 ms 2 1453 26.02 MB/sec warmup 2 sec latency 9.461 ms 2 2294 18.51 MB/sec warmup 3 sec latency 12.940 ms 2 3533 19.19 MB/sec warmup 4 sec latency 10.004 ms 2 4232 16.11 MB/sec warmup 5 sec latency 8.144 ms 2 4994 15.04 MB/sec warmup 6 sec latency 13.382 ms 2 5801 13.12 MB/sec warmup 7 sec latency 9.107 ms 2 7049 14.36 MB/sec warmup 8 sec latency 8.547 ms 2 7730 13.16 MB/sec warmup 9 sec latency 8.630 ms 2 8535 12.85 MB/sec warmup 10 sec latency 10.716 ms 2 9351 11.82 MB/sec warmup 11 sec latency 8.939 ms 2 11242 3.49 MB/sec execute 1 sec latency 10.333 ms 2 12034 6.77 MB/sec execute 2 sec latency 11.219 ms PIDs: 9640 9641 9642 Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 0 Clear changelog after use: no Errors: 0 lustre_rsync took 3 seconds Changelog records consumed: 5008 Resuming dbench 2 12373 2.64 MB/sec execute 5 sec latency 2831.272 ms 2 13226 3.08 MB/sec execute 6 sec latency 10.155 ms 2 14424 5.61 MB/sec execute 7 sec latency 10.024 ms 2 15076 5.18 MB/sec execute 8 sec latency 10.951 ms 2 15861 5.60 MB/sec execute 9 sec latency 12.714 ms 2 16663 5.52 MB/sec execute 10 sec latency 7.893 ms 2 17880 6.83 MB/sec execute 11 sec latency 9.174 ms 2 18526 6.55 MB/sec execute 12 sec latency 12.411 ms 2 19318 6.78 MB/sec execute 13 sec latency 14.664 ms 2 20161 6.57 MB/sec execute 14 sec latency 9.608 ms 2 21350 7.51 MB/sec execute 15 sec latency 8.266 ms Stopping dbench Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 29866 Clear changelog after use: no Errors: 0 lustre_rsync took 1 seconds Changelog records consumed: 3489 Wait for dbench to finish 2 21352 6.70 MB/sec execute 17 sec latency 1829.256 ms 2 22029 6.54 MB/sec execute 18 sec latency 1829.404 ms 2 22799 6.71 MB/sec execute 19 sec latency 10.592 ms 2 23502 6.44 MB/sec execute 20 sec latency 50.086 ms 2 24669 6.96 MB/sec execute 21 sec latency 8.910 ms 2 25343 7.06 MB/sec execute 22 sec latency 8.291 ms 2 26067 6.94 MB/sec execute 23 sec latency 11.710 ms 2 26887 6.94 MB/sec execute 24 sec latency 11.237 ms 2 27872 7.22 MB/sec execute 25 sec latency 9.502 ms 2 28814 7.45 MB/sec execute 26 sec latency 9.483 ms 2 29507 7.32 MB/sec execute 27 sec latency 12.755 ms 2 30285 7.30 MB/sec execute 28 sec latency 9.995 ms 2 31320 7.49 MB/sec execute 29 sec latency 7.965 ms 2 32292 7.71 MB/sec execute 30 sec latency 8.070 ms 2 32943 7.53 MB/sec execute 31 sec latency 12.678 ms 2 33740 7.58 MB/sec execute 32 sec latency 11.763 ms 2 34565 7.51 MB/sec execute 33 sec latency 8.805 ms 2 35732 7.92 MB/sec execute 34 sec latency 9.041 ms 2 36317 7.75 MB/sec execute 35 sec latency 13.191 ms 2 37088 7.79 MB/sec execute 36 sec latency 13.925 ms 2 37905 7.68 MB/sec execute 37 sec latency 10.184 ms 2 39087 8.03 MB/sec execute 38 sec latency 10.893 ms 2 39733 7.91 MB/sec execute 39 sec latency 11.679 ms 2 40468 7.96 MB/sec execute 40 sec latency 9.510 ms 2 41206 7.80 MB/sec execute 41 sec latency 16.485 ms 2 42266 7.97 MB/sec execute 42 sec latency 10.384 ms 2 43040 8.06 MB/sec execute 43 sec latency 10.360 ms 2 43752 7.97 MB/sec execute 44 sec latency 12.300 ms 2 44493 7.94 MB/sec execute 45 sec latency 13.467 ms 2 45476 8.02 MB/sec execute 46 sec latency 10.432 ms 2 46432 8.17 MB/sec execute 47 sec latency 8.302 ms 2 47064 8.05 MB/sec execute 48 sec latency 14.128 ms 2 47845 8.07 MB/sec execute 49 sec latency 8.537 ms 2 48708 8.01 MB/sec execute 50 sec latency 7.743 ms 2 49904 8.28 MB/sec execute 51 sec latency 7.928 ms 2 50508 8.15 MB/sec execute 52 sec latency 10.104 ms 2 51283 8.18 MB/sec execute 53 sec latency 8.862 ms 2 52120 8.10 MB/sec execute 54 sec latency 7.750 ms 2 53345 8.33 MB/sec execute 55 sec latency 7.397 ms 2 53995 8.25 MB/sec execute 56 sec latency 11.038 ms 2 54802 8.27 MB/sec execute 57 sec latency 8.313 ms 2 55645 8.19 MB/sec execute 58 sec latency 7.527 ms 2 56889 8.42 MB/sec execute 59 sec latency 7.871 ms 2 cleanup 60 sec 0 cleanup 60 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 16376 4.336 2831.265 Close 11956 0.769 3.144 Rename 686 5.612 10.881 Unlink 3370 1.395 3.144 Qpathinfo 14786 0.954 6.141 Qfileinfo 2540 0.103 1.091 Qfsinfo 2716 0.063 1.311 Sfileinfo 1310 2.579 14.656 Find 5666 0.290 5.494 WriteX 7988 0.742 6.340 ReadX 25234 0.010 0.234 LockX 52 0.741 0.942 UnlockX 52 0.751 1.004 Flush 1142 4.634 50.078 Throughput 8.41767 MB/sec 2 clients 2 procs max_latency=2831.272 ms stopping dbench in /mnt/lustre/d2b.lustre-rsync-test at Fri May 23 17:45:50 EDT 2025 with return code 0 clean dbench files in /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test /mnt/lustre/d2b.lustre-rsync-test removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients' removed 'client.txt' removed 'dbench' removed 'lib64/libpopt.so.0' removed 'lib64/libc.so.6' removed 'lib64/ld-linux-x86-64.so.2' /mnt/lustre/d2b.lustre-rsync-test dbench successfully finished Starting replication Lustre filesystem: lustre MDT device: lustre-MDT0000 Source: /mnt/lustre Target: /tmp/target Statuslog: /tmp/lustre_rsync.log Changelog registration: cl2 Starting changelog record: 33354 Clear changelog after use: no Errors: 0 lustre_rsync took 10 seconds Changelog records consumed: 14903 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 lustre-MDT0000: changelog user 'cl2' not found PASS 2b (86s) == lustre-rsync-test test complete, duration 182 sec ===== 17:46:02 (1748036762)