== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 21:50:11 (1786758611) Stopping clients: oleg635-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg635-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg635-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg635-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg635-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg635-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg635-server oleg635-server: oleg635-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg635-server' oleg635-server: oleg635-server.virtnet: executing load_modules_local oleg635-server: Loading modules from /home/green/git/lustre-release/lustre oleg635-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg635-server: detected 4 online CPUs by sysfs oleg635-server: MODOPTS_LIBCFS= oleg635-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg635-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg635-server: oleg635-server.virtnet: executing set_default_debug -1 all pdsh@oleg635-client: oleg635-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg635-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg635-server: oleg635-server.virtnet: executing set_default_debug -1 all pdsh@oleg635-client: oleg635-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg635-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg635-server: oleg635-server.virtnet: executing set_default_debug -1 all pdsh@oleg635-client: oleg635-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg635-client.virtnet: -o user_xattr,flock 192.168.206.135@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1005 (time 1786758794.89 total 10.01 last 100.37) - open/close 1931 (time 1786758804.89 total 20.01 last 92.60) - open/close 2957 (time 1786758814.91 total 30.03 last 102.44) - open/close 3771 (time 1786758824.91 total 40.04 last 81.34) total: 4500 open/close in 49.81 seconds: 90.35 ops/second - /unlink 1802 (time 1786758848.72 total 10.00 last 180.11) - /unlink 3116 (time 1786758858.72 total 20.01 last 131.33) total: 4500 /unlink in 28.31 seconds: 158.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 943 (time 1786758881.19 total 10.00 last 94.27) - open/close 1853 (time 1786758891.20 total 20.01 last 90.92) - open/close 2805 (time 1786758901.20 total 30.02 last 95.13) - open/close 3743 (time 1786758911.23 total 40.04 last 93.58) total: 4500 open/close in 47.69 seconds: 94.36 ops/second - /unlink 1788 (time 1786758932.70 total 10.00 last 178.77) - /unlink 3447 (time 1786758942.70 total 20.00 last 165.87) total: 4500 /unlink in 27.36 seconds: 164.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 919 (time 1786758964.90 total 10.01 last 91.84) - open/close 1818 (time 1786758974.93 total 20.04 last 89.61) - open/close 2781 (time 1786758984.93 total 30.04 last 96.29) - open/close 3581 (time 1786758994.94 total 40.05 last 79.96) - open/close 4379 (time 1786759004.94 total 50.05 last 79.77) total: 4500 open/close in 51.55 seconds: 87.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1666 (time 1786759020.75 total 10.01 last 166.51) - /unlink 3159 (time 1786759030.76 total 20.01 last 149.20) total: 4500 /unlink in 29.63 seconds: 151.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 775 (time 1786759055.93 total 10.01 last 77.44) - open/close 1577 (time 1786759065.93 total 20.01 last 80.17) - open/close 2553 (time 1786759075.94 total 30.02 last 97.57) - open/close 3295 (time 1786759085.94 total 40.02 last 74.16) - open/close 4238 (time 1786759095.94 total 50.03 last 94.25) total: 4500 open/close in 52.54 seconds: 85.65 ops/second - /unlink 1690 (time 1786759112.27 total 10.00 last 168.95) - /unlink 3214 (time 1786759122.27 total 20.00 last 152.39) total: 4500 /unlink in 27.88 seconds: 161.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1007 (time 1786759144.40 total 10.00 last 100.67) - open/close 1883 (time 1786759154.41 total 20.00 last 87.59) - open/close 2871 (time 1786759164.42 total 30.02 last 98.61) - open/close 3757 (time 1786759174.43 total 40.03 last 88.55) total: 4500 open/close in 47.68 seconds: 94.37 ops/second - /unlink 1811 (time 1786759197.26 total 10.00 last 181.06) - /unlink 3527 (time 1786759207.26 total 20.01 last 171.55) total: 4500 /unlink in 26.23 seconds: 171.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 889 (time 1786759228.62 total 10.00 last 88.89) - open/close 1806 (time 1786759238.63 total 20.01 last 91.61) - open/close 2754 (time 1786759248.63 total 30.01 last 94.77) - open/close 3708 (time 1786759258.63 total 40.02 last 95.36) total: 4500 open/close in 48.37 seconds: 93.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1644 (time 1786759281.35 total 10.01 last 164.30) - /unlink 3023 (time 1786759291.35 total 20.01 last 137.86) total: 4500 /unlink in 29.17 seconds: 154.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 948 (time 1786759315.18 total 10.00 last 94.77) - open/close 1879 (time 1786759325.18 total 20.01 last 93.07) - open/close 2762 (time 1786759335.19 total 30.02 last 88.21) - open/close 3670 (time 1786759345.20 total 40.02 last 90.76) total: 4500 open/close in 49.63 seconds: 90.68 ops/second - /unlink 1643 (time 1786759369.13 total 10.00 last 164.26) - /unlink 3119 (time 1786759379.13 total 20.00 last 147.57) total: 4500 /unlink in 28.41 seconds: 158.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 582 (time 1786759403.79 total 10.02 last 58.08) - open/close 1293 (time 1786759413.81 total 20.04 last 70.94) - open/close 1975 (time 1786759423.82 total 30.05 last 68.16) - open/close 2581 (time 1786759433.82 total 40.05 last 60.59) - open/close 3396 (time 1786759443.82 total 50.06 last 81.46) - open/close 4330 (time 1786759453.83 total 60.06 last 93.33) total: 4500 open/close in 62.94 seconds: 71.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1303 (time 1786759472.98 total 10.02 last 130.04) - /unlink 2811 (time 1786759482.98 total 20.02 last 150.75) - /unlink 4305 (time 1786759492.98 total 30.02 last 149.38) total: 4500 /unlink in 31.25 seconds: 144.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 854 (time 1786759508.32 total 10.00 last 85.37) - open/close 1824 (time 1786759518.32 total 20.01 last 96.96) - open/close 2747 (time 1786759528.33 total 30.01 last 92.25) - open/close 3605 (time 1786759538.33 total 40.02 last 85.76) total: 4500 open/close in 49.87 seconds: 90.23 ops/second - /unlink 1749 (time 1786759562.70 total 10.00 last 174.89) - /unlink 3324 (time 1786759572.71 total 20.01 last 157.35) - /unlink 4414 (time 1786759582.71 total 30.01 last 109.00) total: 4500 /unlink in 31.05 seconds: 144.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 610 (time 1786759599.81 total 10.01 last 60.91) - open/close 1149 (time 1786759609.81 total 20.02 last 53.89) - open/close 1954 (time 1786759619.82 total 30.02 last 80.48) - open/close 2603 (time 1786759629.86 total 40.06 last 64.63) - open/close 3324 (time 1786759639.86 total 50.07 last 72.05) - open/close 4064 (time 1786759649.87 total 60.07 last 73.99) total: 4500 open/close in 65.67 seconds: 68.52 ops/second - /unlink 1702 (time 1786759670.11 total 10.00 last 170.15) - /unlink 3228 (time 1786759680.13 total 20.02 last 152.33) total: 4500 /unlink in 28.81 seconds: 156.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 861 (time 1786759704.55 total 10.00 last 86.08) - open/close 1769 (time 1786759714.56 total 20.01 last 90.71) - open/close 2526 (time 1786759724.57 total 30.02 last 75.65) - open/close 3211 (time 1786759734.58 total 40.03 last 68.41) - open/close 3979 (time 1786759744.58 total 50.04 last 76.77) total: 4500 open/close in 56.43 seconds: 79.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1678 (time 1786759766.30 total 10.00 last 167.75) - /unlink 3345 (time 1786759776.31 total 20.01 last 166.61) total: 4500 /unlink in 26.69 seconds: 168.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 756 (time 1786759797.77 total 10.01 last 75.51) - open/close 1581 (time 1786759807.77 total 20.01 last 82.48) - open/close 2214 (time 1786759817.78 total 30.02 last 63.25) - open/close 2921 (time 1786759827.80 total 40.05 last 70.54) - open/close 3656 (time 1786759837.81 total 50.05 last 73.46) - open/close 4460 (time 1786759847.82 total 60.06 last 80.33) total: 4500 open/close in 60.76 seconds: 74.07 ops/second - /unlink 1516 (time 1786759864.31 total 10.00 last 151.56) - /unlink 2884 (time 1786759874.32 total 20.01 last 136.75) - /unlink 4402 (time 1786759884.33 total 30.02 last 151.60) total: 4500 /unlink in 31.53 seconds: 142.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 853 (time 1786759901.12 total 10.01 last 85.25) - open/close 1713 (time 1786759911.13 total 20.01 last 85.93) - open/close 2594 (time 1786759921.13 total 30.02 last 88.06) - open/close 3444 (time 1786759931.14 total 40.02 last 84.96) - open/close 4372 (time 1786759941.14 total 50.03 last 92.78) total: 4500 open/close in 51.31 seconds: 87.70 ops/second - /unlink 1728 (time 1786759956.36 total 10.00 last 172.80) - /unlink 3311 (time 1786759966.36 total 20.00 last 158.23) total: 4500 /unlink in 27.32 seconds: 164.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 894 (time 1786759987.91 total 10.00 last 89.38) - open/close 1843 (time 1786759997.92 total 20.01 last 94.82) - open/close 2439 (time 1786760007.94 total 30.03 last 59.48) - open/close 2979 (time 1786760017.95 total 40.04 last 53.95) - open/close 3450 (time 1786760027.95 total 50.04 last 47.08) - open/close 4143 (time 1786760037.95 total 60.05 last 69.28) total: 4500 open/close in 64.35 seconds: 69.93 ops/second - /unlink 1576 (time 1786760056.46 total 10.01 last 157.52) - /unlink 3198 (time 1786760066.46 total 20.01 last 162.13) total: 4500 /unlink in 28.63 seconds: 157.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 885 (time 1786760090.42 total 10.01 last 88.45) - open/close 1804 (time 1786760100.42 total 20.01 last 91.86) - open/close 2768 (time 1786760110.43 total 30.01 last 96.36) - open/close 3721 (time 1786760120.43 total 40.02 last 95.24) total: 4500 open/close in 48.91 seconds: 92.00 ops/second - /unlink 1793 (time 1786760143.79 total 10.01 last 179.16) - /unlink 3396 (time 1786760153.79 total 20.01 last 160.25) total: 4500 /unlink in 27.26 seconds: 165.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 885 (time 1786760176.27 total 10.01 last 88.44) - open/close 1648 (time 1786760186.28 total 20.02 last 76.19) - open/close 2578 (time 1786760196.29 total 30.03 last 92.91) - open/close 3499 (time 1786760206.31 total 40.05 last 91.94) - open/close 4445 (time 1786760216.31 total 50.05 last 94.54) total: 4500 open/close in 50.72 seconds: 88.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1623 (time 1786760230.74 total 10.00 last 162.29) - /unlink 3024 (time 1786760240.74 total 20.00 last 140.06) - /unlink 4350 (time 1786760250.75 total 30.01 last 132.47) total: 4500 /unlink in 31.26 seconds: 143.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 632 (time 1786760266.46 total 10.00 last 63.18) - open/close 1309 (time 1786760276.47 total 20.01 last 67.65) - open/close 2293 (time 1786760286.47 total 30.02 last 98.35) - open/close 3230 (time 1786760296.47 total 40.02 last 93.68) - open/close 4143 (time 1786760306.48 total 50.02 last 91.25) total: 4500 open/close in 53.82 seconds: 83.61 ops/second - /unlink 1624 (time 1786760323.92 total 10.01 last 162.29) - /unlink 3230 (time 1786760333.92 total 20.01 last 160.48) total: 4500 /unlink in 28.42 seconds: 158.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 860 (time 1786760356.78 total 10.00 last 85.96) - open/close 1631 (time 1786760366.79 total 20.01 last 77.03) - open/close 2210 (time 1786760376.79 total 30.02 last 57.86) - open/close 2808 (time 1786760386.80 total 40.03 last 59.76) - open/close 3390 (time 1786760396.80 total 50.03 last 58.19) - open/close 4266 (time 1786760406.81 total 60.03 last 87.55) total: 4500 open/close in 62.41 seconds: 72.10 ops/second - /unlink 1700 (time 1786760423.20 total 10.00 last 169.99) - /unlink 3291 (time 1786760433.21 total 20.01 last 158.93) total: 4500 /unlink in 27.57 seconds: 163.19 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95221 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg635-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg635-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg635-server stop mds service on oleg635-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg635-server stop mds service on oleg635-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg635-server unloading modules via unload_modules_local on: 'oleg635-server' oleg635-server: oleg635-server.virtnet: executing unload_modules_local oleg635-server: modules unloaded.