== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 02:16:14 (1786169774) Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg340-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg340-server oleg340-server: oleg340-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg340-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 889 (time 1786169971.32 total 10.00 last 88.89) - open/close 1724 (time 1786169981.32 total 20.00 last 83.48) - open/close 2568 (time 1786169991.32 total 30.01 last 84.39) - open/close 3390 (time 1786170001.33 total 40.01 last 82.16) - open/close 4285 (time 1786170011.33 total 50.02 last 89.44) total: 4500 open/close in 52.68 seconds: 85.42 ops/second - /unlink 1613 (time 1786170028.31 total 10.01 last 161.12) - /unlink 3050 (time 1786170038.31 total 20.01 last 143.67) total: 4500 /unlink in 28.50 seconds: 157.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 968 (time 1786170060.54 total 10.00 last 96.76) - open/close 1840 (time 1786170070.57 total 20.04 last 86.91) - open/close 2747 (time 1786170080.70 total 30.17 last 89.54) - open/close 3580 (time 1786170090.72 total 40.18 last 83.16) - open/close 4445 (time 1786170100.72 total 50.19 last 86.48) total: 4500 open/close in 50.86 seconds: 88.48 ops/second - /unlink 1617 (time 1786170115.64 total 10.00 last 161.70) - /unlink 2962 (time 1786170125.65 total 20.01 last 134.39) total: 4500 /unlink in 29.90 seconds: 150.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 915 (time 1786170151.03 total 10.00 last 91.46) - open/close 1868 (time 1786170161.07 total 20.05 last 94.87) - open/close 2785 (time 1786170171.07 total 30.05 last 91.69) - open/close 3687 (time 1786170181.07 total 40.05 last 90.19) total: 4500 open/close in 49.70 seconds: 90.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1399 (time 1786170205.54 total 10.00 last 139.83) - /unlink 2996 (time 1786170215.54 total 20.01 last 159.68) - /unlink 4415 (time 1786170225.55 total 30.02 last 141.75) total: 4500 /unlink in 31.02 seconds: 145.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 939 (time 1786170240.97 total 10.01 last 93.82) - open/close 1799 (time 1786170250.97 total 20.01 last 85.99) - open/close 2650 (time 1786170260.97 total 30.01 last 85.09) - open/close 3483 (time 1786170270.97 total 40.01 last 83.30) - open/close 4353 (time 1786170280.98 total 50.02 last 86.96) total: 4500 open/close in 51.51 seconds: 87.35 ops/second - /unlink 1560 (time 1786170296.17 total 10.01 last 155.80) - /unlink 2965 (time 1786170306.17 total 20.02 last 140.46) - /unlink 4471 (time 1786170316.17 total 30.02 last 150.57) total: 4500 /unlink in 30.25 seconds: 148.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 882 (time 1786170330.52 total 10.00 last 88.18) - open/close 1755 (time 1786170340.52 total 20.00 last 87.29) - open/close 2598 (time 1786170350.55 total 30.02 last 84.12) - open/close 3580 (time 1786170360.55 total 40.03 last 98.13) total: 4500 open/close in 50.02 seconds: 89.96 ops/second - /unlink 1634 (time 1786170384.81 total 10.00 last 163.34) - /unlink 3146 (time 1786170394.82 total 20.01 last 151.16) total: 4500 /unlink in 29.51 seconds: 152.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 867 (time 1786170418.83 total 10.00 last 86.70) - open/close 1714 (time 1786170428.83 total 20.00 last 84.68) - open/close 2647 (time 1786170438.84 total 30.01 last 93.25) - open/close 3451 (time 1786170448.85 total 40.02 last 80.34) - open/close 4316 (time 1786170458.88 total 50.05 last 86.19) total: 4500 open/close in 52.05 seconds: 86.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1573 (time 1786170475.68 total 10.00 last 157.22) - /unlink 2982 (time 1786170485.69 total 20.01 last 140.82) total: 4500 /unlink in 29.95 seconds: 150.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 893 (time 1786170510.12 total 10.01 last 89.23) - open/close 1755 (time 1786170520.12 total 20.01 last 86.15) - open/close 2707 (time 1786170530.13 total 30.02 last 95.18) - open/close 3656 (time 1786170540.13 total 40.02 last 94.85) total: 4500 open/close in 49.21 seconds: 91.44 ops/second - /unlink 1534 (time 1786170562.70 total 10.00 last 153.33) - /unlink 3183 (time 1786170572.70 total 20.00 last 164.90) total: 4500 /unlink in 27.84 seconds: 161.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 911 (time 1786170595.68 total 10.00 last 91.09) - open/close 1781 (time 1786170605.76 total 20.09 last 86.25) - open/close 2666 (time 1786170615.77 total 30.09 last 88.44) - open/close 3633 (time 1786170625.78 total 40.10 last 96.65) total: 4500 open/close in 49.87 seconds: 90.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1658 (time 1786170649.76 total 10.00 last 165.78) - /unlink 3312 (time 1786170659.76 total 20.00 last 165.35) total: 4500 /unlink in 28.01 seconds: 160.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 973 (time 1786170682.88 total 10.02 last 97.08) - open/close 1764 (time 1786170692.88 total 20.03 last 79.06) - open/close 2712 (time 1786170702.89 total 30.04 last 94.70) - open/close 3572 (time 1786170712.89 total 40.04 last 85.97) - open/close 4471 (time 1786170722.89 total 50.04 last 89.90) total: 4500 open/close in 50.28 seconds: 89.49 ops/second - /unlink 1524 (time 1786170736.98 total 10.00 last 152.40) - /unlink 2953 (time 1786170746.99 total 20.00 last 142.87) total: 4500 /unlink in 29.36 seconds: 153.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 892 (time 1786170771.04 total 10.00 last 89.19) - open/close 1731 (time 1786170781.05 total 20.01 last 83.85) - open/close 2569 (time 1786170791.06 total 30.02 last 83.69) - open/close 3454 (time 1786170801.07 total 40.02 last 88.48) - open/close 4294 (time 1786170811.07 total 50.02 last 83.99) total: 4500 open/close in 52.29 seconds: 86.05 ops/second - /unlink 1606 (time 1786170827.93 total 10.00 last 160.53) - /unlink 3025 (time 1786170837.93 total 20.01 last 141.85) total: 4500 /unlink in 29.60 seconds: 152.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 914 (time 1786170861.94 total 10.00 last 91.39) - open/close 1666 (time 1786170871.94 total 20.01 last 75.16) - open/close 2593 (time 1786170881.95 total 30.01 last 92.64) - open/close 3480 (time 1786170891.95 total 40.02 last 88.67) - open/close 4407 (time 1786170901.96 total 50.02 last 92.61) total: 4500 open/close in 51.00 seconds: 88.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1642 (time 1786170917.14 total 10.00 last 164.17) - /unlink 3205 (time 1786170927.16 total 20.02 last 155.97) total: 4500 /unlink in 28.00 seconds: 160.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 984 (time 1786170949.77 total 10.01 last 98.34) - open/close 1897 (time 1786170959.77 total 20.01 last 91.28) - open/close 2791 (time 1786170969.77 total 30.01 last 89.37) - open/close 3707 (time 1786170979.78 total 40.01 last 91.57) total: 4500 open/close in 48.37 seconds: 93.04 ops/second - /unlink 1544 (time 1786171001.91 total 10.00 last 154.36) - /unlink 2972 (time 1786171011.91 total 20.00 last 142.79) - /unlink 4397 (time 1786171021.91 total 30.00 last 142.50) total: 4500 /unlink in 30.78 seconds: 146.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 762 (time 1786171038.61 total 10.00 last 76.19) - open/close 1674 (time 1786171048.62 total 20.01 last 91.11) - open/close 2544 (time 1786171058.63 total 30.02 last 86.92) - open/close 3452 (time 1786171068.65 total 40.04 last 90.58) - open/close 4389 (time 1786171078.69 total 50.08 last 93.35) total: 4500 open/close in 51.11 seconds: 88.05 ops/second - /unlink 1639 (time 1786171094.63 total 10.01 last 163.81) - /unlink 3215 (time 1786171104.64 total 20.01 last 157.49) total: 4500 /unlink in 28.81 seconds: 156.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 795 (time 1786171128.53 total 10.01 last 79.46) - open/close 1647 (time 1786171138.54 total 20.01 last 85.16) - open/close 2580 (time 1786171148.58 total 30.05 last 92.92) - open/close 3486 (time 1786171158.58 total 40.05 last 90.59) - open/close 4374 (time 1786171168.58 total 50.06 last 88.75) total: 4500 open/close in 51.38 seconds: 87.58 ops/second - /unlink 1290 (time 1786171183.85 total 10.01 last 128.85) - /unlink 2699 (time 1786171193.85 total 20.01 last 140.89) - /unlink 4025 (time 1786171203.86 total 30.02 last 132.55) total: 4500 /unlink in 33.76 seconds: 133.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 804 (time 1786171221.33 total 10.00 last 80.38) - open/close 1655 (time 1786171231.33 total 20.00 last 85.08) - open/close 2526 (time 1786171241.34 total 30.01 last 87.01) - open/close 3397 (time 1786171251.35 total 40.02 last 87.01) - open/close 4207 (time 1786171261.36 total 50.03 last 80.93) total: 4500 open/close in 53.31 seconds: 84.42 ops/second - /unlink 1492 (time 1786171278.29 total 10.01 last 149.05) - /unlink 3002 (time 1786171288.29 total 20.01 last 150.93) - /unlink 4448 (time 1786171298.30 total 30.02 last 144.57) total: 4500 /unlink in 30.51 seconds: 147.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 875 (time 1786171313.57 total 10.00 last 87.49) - open/close 1731 (time 1786171323.57 total 20.00 last 85.58) - open/close 2648 (time 1786171333.58 total 30.01 last 91.67) - open/close 3494 (time 1786171343.59 total 40.02 last 84.45) - open/close 4369 (time 1786171353.60 total 50.03 last 87.47) total: 4500 open/close in 51.78 seconds: 86.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1519 (time 1786171369.87 total 10.00 last 151.84) - /unlink 2670 (time 1786171379.87 total 20.01 last 115.07) - /unlink 4011 (time 1786171389.89 total 30.02 last 133.93) total: 4500 /unlink in 33.46 seconds: 134.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 787 (time 1786171410.47 total 10.01 last 78.63) - open/close 1617 (time 1786171420.47 total 20.01 last 82.99) - open/close 2479 (time 1786171430.47 total 30.01 last 86.17) - open/close 3264 (time 1786171440.48 total 40.02 last 78.46) - open/close 4166 (time 1786171450.48 total 50.03 last 90.13) total: 4500 open/close in 53.84 seconds: 83.59 ops/second - /unlink 1484 (time 1786171468.64 total 10.00 last 148.38) - /unlink 2863 (time 1786171478.64 total 20.00 last 137.86) - /unlink 4383 (time 1786171488.65 total 30.01 last 151.94) total: 4500 /unlink in 30.96 seconds: 145.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 817 (time 1786171503.73 total 10.01 last 81.65) - open/close 1652 (time 1786171513.74 total 20.01 last 83.43) - open/close 2563 (time 1786171523.75 total 30.02 last 91.03) - open/close 3443 (time 1786171533.75 total 40.02 last 87.98) - open/close 4332 (time 1786171543.75 total 50.03 last 88.85) total: 4500 open/close in 52.33 seconds: 85.99 ops/second - /unlink 1673 (time 1786171560.06 total 10.01 last 167.11) - /unlink 3200 (time 1786171570.07 total 20.02 last 152.61) total: 4500 /unlink in 28.31 seconds: 158.96 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95187 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server unloading modules via unload_modules_local on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: modules unloaded.