== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:24:28 (1786764268) Stopping clients: oleg102-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg102-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg102-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg102-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg102-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg102-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg102-server oleg102-server: oleg102-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg102-server' oleg102-server: oleg102-server.virtnet: executing load_modules_local oleg102-server: Loading modules from /home/green/git/lustre-release/lustre oleg102-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg102-server: detected 4 online CPUs by sysfs oleg102-server: MODOPTS_LIBCFS= oleg102-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg102-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg102-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg102-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg102-client.virtnet: -o user_xattr,flock 192.168.201.102@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1040 (time 1786764464.99 total 10.01 last 103.92) - open/close 1995 (time 1786764475.01 total 20.03 last 95.31) - open/close 2989 (time 1786764485.02 total 30.04 last 99.26) - open/close 3946 (time 1786764495.04 total 40.06 last 95.57) total: 4500 open/close in 46.24 seconds: 97.33 ops/second - /unlink 1801 (time 1786764515.53 total 10.00 last 180.10) - /unlink 3728 (time 1786764525.53 total 20.00 last 192.66) total: 4500 /unlink in 24.44 seconds: 184.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 976 (time 1786764544.51 total 10.00 last 97.60) - open/close 1898 (time 1786764554.51 total 20.01 last 92.12) - open/close 2871 (time 1786764564.51 total 30.01 last 97.29) - open/close 3645 (time 1786764574.53 total 40.03 last 77.27) total: 4500 open/close in 49.55 seconds: 90.81 ops/second - /unlink 1606 (time 1786764599.53 total 10.01 last 160.51) - /unlink 3283 (time 1786764609.53 total 20.01 last 167.66) total: 4500 /unlink in 27.03 seconds: 166.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 905 (time 1786764630.52 total 10.01 last 90.45) - open/close 1760 (time 1786764640.53 total 20.01 last 85.44) - open/close 2712 (time 1786764650.53 total 30.01 last 95.19) - open/close 3481 (time 1786764660.54 total 40.02 last 76.85) - open/close 4411 (time 1786764670.54 total 50.02 last 92.97) total: 4500 open/close in 50.93 seconds: 88.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1650 (time 1786764685.62 total 10.01 last 164.88) - /unlink 3111 (time 1786764695.62 total 20.01 last 146.09) total: 4500 /unlink in 28.49 seconds: 157.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 944 (time 1786764719.14 total 10.00 last 94.38) - open/close 1968 (time 1786764729.18 total 20.05 last 101.93) - open/close 2833 (time 1786764739.19 total 30.05 last 86.49) - open/close 3757 (time 1786764749.19 total 40.05 last 92.40) total: 4500 open/close in 47.28 seconds: 95.19 ops/second - /unlink 1798 (time 1786764770.16 total 10.00 last 179.75) - /unlink 3408 (time 1786764780.16 total 20.00 last 161.00) total: 4500 /unlink in 27.40 seconds: 164.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1075 (time 1786764801.90 total 10.01 last 107.42) - open/close 2032 (time 1786764811.90 total 20.01 last 95.69) - open/close 2909 (time 1786764821.92 total 30.03 last 87.47) - open/close 3657 (time 1786764831.93 total 40.04 last 74.78) - open/close 4486 (time 1786764841.94 total 50.05 last 82.82) total: 4500 open/close in 50.17 seconds: 89.69 ops/second - /unlink 1869 (time 1786764856.57 total 10.01 last 186.72) - /unlink 3629 (time 1786764866.57 total 20.01 last 175.95) total: 4500 /unlink in 24.97 seconds: 180.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 931 (time 1786764885.81 total 10.00 last 93.09) - open/close 1894 (time 1786764895.81 total 20.00 last 96.29) - open/close 2823 (time 1786764905.82 total 30.01 last 92.83) - open/close 3633 (time 1786764915.82 total 40.02 last 80.93) - open/close 4498 (time 1786764925.83 total 50.03 last 86.43) total: 4500 open/close in 50.08 seconds: 89.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1319 (time 1786764945.10 total 10.00 last 131.87) - /unlink 2672 (time 1786764955.11 total 20.01 last 135.18) - /unlink 3969 (time 1786764965.11 total 30.02 last 129.63) total: 4500 /unlink in 33.10 seconds: 135.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 902 (time 1786764982.15 total 10.02 last 90.06) - open/close 1871 (time 1786764992.17 total 20.03 last 96.73) - open/close 2799 (time 1786765002.18 total 30.04 last 92.75) - open/close 3602 (time 1786765012.18 total 40.04 last 80.29) - open/close 4407 (time 1786765022.18 total 50.04 last 80.47) total: 4500 open/close in 51.14 seconds: 88.00 ops/second - /unlink 1876 (time 1786765039.08 total 10.00 last 187.58) - /unlink 3450 (time 1786765049.09 total 20.01 last 157.29) total: 4500 /unlink in 27.93 seconds: 161.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 946 (time 1786765071.77 total 10.01 last 94.49) - open/close 1937 (time 1786765081.77 total 20.01 last 99.07) - open/close 2920 (time 1786765091.78 total 30.02 last 98.20) - open/close 3925 (time 1786765101.79 total 40.04 last 100.36) total: 4500 open/close in 46.33 seconds: 97.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1689 (time 1786765122.21 total 10.00 last 168.86) - /unlink 3147 (time 1786765132.21 total 20.01 last 145.74) total: 4500 /unlink in 27.69 seconds: 162.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 971 (time 1786765153.36 total 10.00 last 97.06) - open/close 1893 (time 1786765163.37 total 20.01 last 92.11) - open/close 2775 (time 1786765173.38 total 30.03 last 88.07) - open/close 3652 (time 1786765183.39 total 40.04 last 87.59) total: 4500 open/close in 49.60 seconds: 90.73 ops/second - /unlink 1626 (time 1786765207.39 total 10.00 last 162.54) - /unlink 3227 (time 1786765217.40 total 20.01 last 160.05) total: 4500 /unlink in 29.39 seconds: 153.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 830 (time 1786765241.72 total 10.01 last 82.94) - open/close 1630 (time 1786765251.73 total 20.01 last 79.95) - open/close 2522 (time 1786765261.73 total 30.02 last 89.18) - open/close 3450 (time 1786765271.74 total 40.02 last 92.72) - open/close 4240 (time 1786765281.74 total 50.02 last 78.99) total: 4500 open/close in 52.86 seconds: 85.13 ops/second - /unlink 1848 (time 1786765299.42 total 10.00 last 184.75) - /unlink 3535 (time 1786765309.43 total 20.00 last 168.68) total: 4500 /unlink in 25.74 seconds: 174.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1024 (time 1786765329.49 total 10.00 last 102.36) - open/close 1933 (time 1786765339.50 total 20.01 last 90.83) - open/close 2895 (time 1786765349.51 total 30.02 last 96.12) - open/close 3686 (time 1786765359.52 total 40.03 last 79.01) - open/close 4461 (time 1786765369.53 total 50.04 last 77.47) total: 4500 open/close in 50.54 seconds: 89.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1828 (time 1786765384.08 total 10.00 last 182.72) - /unlink 3595 (time 1786765394.08 total 20.01 last 176.68) total: 4500 /unlink in 25.35 seconds: 177.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 863 (time 1786765413.85 total 10.01 last 86.24) - open/close 1705 (time 1786765423.87 total 20.03 last 83.98) - open/close 2649 (time 1786765433.89 total 30.05 last 94.22) - open/close 3536 (time 1786765443.90 total 40.06 last 88.67) - open/close 4297 (time 1786765453.91 total 50.07 last 76.03) total: 4500 open/close in 53.49 seconds: 84.12 ops/second - /unlink 1390 (time 1786765472.33 total 10.00 last 138.95) - /unlink 2838 (time 1786765482.33 total 20.00 last 144.79) - /unlink 4493 (time 1786765492.34 total 30.01 last 165.38) total: 4500 /unlink in 30.05 seconds: 149.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1017 (time 1786765506.72 total 10.00 last 101.66) - open/close 1856 (time 1786765516.73 total 20.01 last 83.86) - open/close 2767 (time 1786765526.73 total 30.01 last 91.08) - open/close 3492 (time 1786765536.73 total 40.01 last 72.49) - open/close 4385 (time 1786765546.74 total 50.02 last 89.25) total: 4500 open/close in 51.25 seconds: 87.81 ops/second - /unlink 1426 (time 1786765563.53 total 10.01 last 142.50) - /unlink 2810 (time 1786765573.53 total 20.01 last 138.38) - /unlink 4285 (time 1786765583.53 total 30.01 last 147.46) total: 4500 /unlink in 31.98 seconds: 140.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 772 (time 1786765600.96 total 10.01 last 77.16) - open/close 1659 (time 1786765610.96 total 20.01 last 88.66) - open/close 2533 (time 1786765620.97 total 30.02 last 87.34) - open/close 3303 (time 1786765630.99 total 40.04 last 76.85) - open/close 4069 (time 1786765641.03 total 50.07 last 76.33) total: 4500 open/close in 54.74 seconds: 82.20 ops/second - /unlink 1571 (time 1786765660.01 total 10.00 last 157.10) - /unlink 3060 (time 1786765670.01 total 20.00 last 148.88) total: 4500 /unlink in 28.29 seconds: 159.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 807 (time 1786765693.28 total 10.00 last 80.68) - open/close 1598 (time 1786765703.31 total 20.03 last 78.88) - open/close 2427 (time 1786765713.34 total 30.05 last 82.70) - open/close 3315 (time 1786765723.34 total 40.06 last 88.74) - open/close 4230 (time 1786765733.34 total 50.06 last 91.50) total: 4500 open/close in 53.13 seconds: 84.70 ops/second - /unlink 1700 (time 1786765750.68 total 10.00 last 169.94) - /unlink 3282 (time 1786765760.69 total 20.01 last 158.14) total: 4500 /unlink in 28.04 seconds: 160.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 971 (time 1786765783.14 total 10.01 last 97.04) - open/close 1868 (time 1786765793.15 total 20.01 last 89.65) - open/close 2752 (time 1786765803.16 total 30.02 last 88.29) - open/close 3602 (time 1786765813.17 total 40.03 last 84.97) total: 4500 open/close in 49.32 seconds: 91.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1582 (time 1786765836.01 total 10.00 last 158.19) - /unlink 3238 (time 1786765846.01 total 20.00 last 165.57) total: 4500 /unlink in 27.85 seconds: 161.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 824 (time 1786765867.80 total 10.00 last 82.39) - open/close 1662 (time 1786765877.80 total 20.01 last 83.76) - open/close 2457 (time 1786765887.81 total 30.01 last 79.45) - open/close 3318 (time 1786765897.81 total 40.01 last 86.09) - open/close 4022 (time 1786765907.81 total 50.01 last 70.39) total: 4500 open/close in 56.84 seconds: 79.18 ops/second - /unlink 1636 (time 1786765928.59 total 10.00 last 163.59) - /unlink 3032 (time 1786765938.59 total 20.01 last 139.52) total: 4500 /unlink in 28.27 seconds: 159.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1010 (time 1786765962.20 total 10.00 last 100.97) - open/close 1915 (time 1786765972.21 total 20.01 last 90.39) - open/close 2752 (time 1786765982.21 total 30.02 last 83.69) - open/close 3627 (time 1786765992.22 total 40.02 last 87.44) total: 4500 open/close in 49.03 seconds: 91.78 ops/second - /unlink 1756 (time 1786766014.58 total 10.00 last 175.54) - /unlink 3523 (time 1786766024.58 total 20.00 last 176.70) total: 4500 /unlink in 25.60 seconds: 175.77 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95088 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg102-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg102-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg102-server stop mds service on oleg102-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg102-server stop mds service on oleg102-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg102-server unloading modules via unload_modules_local on: 'oleg102-server' oleg102-server: oleg102-server.virtnet: executing unload_modules_local oleg102-server: modules unloaded.