== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 06:25:28 (1778927128) Stopping clients: oleg246-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg246-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg246-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg246-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg246-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg246-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg246-server oleg246-server: oleg246-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg246-server' oleg246-server: oleg246-server.virtnet: executing load_modules_local oleg246-server: Loading modules from /home/green/git/lustre-release/lustre oleg246-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg246-server: detected 4 online CPUs by sysfs oleg246-server: MODOPTS_LIBCFS= oleg246-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg246-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg246-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg246-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg246-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg246-client.virtnet: -o user_xattr,flock 192.168.202.146@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 947 (time 1778927284.46 total 10.00 last 94.66) - open/close 1787 (time 1778927294.46 total 20.01 last 83.96) - open/close 2634 (time 1778927304.47 total 30.01 last 84.68) - open/close 3382 (time 1778927314.47 total 40.01 last 74.80) - open/close 4256 (time 1778927324.47 total 50.02 last 87.35) total: 4500 open/close in 53.20 seconds: 84.59 ops/second - /unlink 1722 (time 1778927340.62 total 10.00 last 172.15) - /unlink 3317 (time 1778927350.63 total 20.01 last 159.42) total: 4500 /unlink in 27.78 seconds: 162.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 858 (time 1778927372.00 total 10.01 last 85.73) - open/close 1709 (time 1778927382.00 total 20.01 last 85.07) - open/close 2548 (time 1778927392.01 total 30.02 last 83.79) - open/close 3329 (time 1778927402.02 total 40.03 last 78.04) - open/close 4201 (time 1778927412.03 total 50.04 last 87.11) total: 4500 open/close in 53.13 seconds: 84.70 ops/second - /unlink 1612 (time 1778927428.28 total 10.00 last 161.16) - /unlink 3321 (time 1778927438.28 total 20.00 last 170.87) total: 4500 /unlink in 27.28 seconds: 164.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 862 (time 1778927458.65 total 10.00 last 86.17) - open/close 1767 (time 1778927468.66 total 20.01 last 90.40) - open/close 2641 (time 1778927478.67 total 30.03 last 87.30) - open/close 3567 (time 1778927488.68 total 40.04 last 92.50) - open/close 4389 (time 1778927498.69 total 50.04 last 82.18) total: 4500 open/close in 51.43 seconds: 87.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1658 (time 1778927513.26 total 10.00 last 165.75) - /unlink 3120 (time 1778927523.26 total 20.00 last 146.18) total: 4500 /unlink in 28.65 seconds: 157.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 975 (time 1778927545.74 total 10.01 last 97.43) - open/close 1870 (time 1778927555.74 total 20.01 last 89.46) - open/close 2792 (time 1778927565.76 total 30.02 last 92.08) - open/close 3666 (time 1778927575.76 total 40.03 last 87.35) total: 4500 open/close in 48.26 seconds: 93.25 ops/second - /unlink 1906 (time 1778927596.62 total 10.00 last 190.59) - /unlink 4026 (time 1778927606.62 total 20.00 last 211.97) total: 4500 /unlink in 22.21 seconds: 202.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 978 (time 1778927621.77 total 10.00 last 97.79) - open/close 1975 (time 1778927631.77 total 20.01 last 99.64) - open/close 2864 (time 1778927641.78 total 30.01 last 88.87) - open/close 3859 (time 1778927651.79 total 40.03 last 99.34) total: 4500 open/close in 45.73 seconds: 98.39 ops/second - /unlink 1924 (time 1778927670.20 total 10.00 last 192.39) - /unlink 3945 (time 1778927680.20 total 20.00 last 202.03) total: 4500 /unlink in 22.80 seconds: 197.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1004 (time 1778927695.61 total 10.02 last 100.23) - open/close 2113 (time 1778927705.61 total 20.02 last 110.89) - open/close 3307 (time 1778927715.62 total 30.02 last 119.36) - open/close 4365 (time 1778927725.62 total 40.03 last 105.76) total: 4500 open/close in 41.32 seconds: 108.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2040 (time 1778927740.24 total 10.01 last 203.89) - /unlink 3888 (time 1778927750.24 total 20.01 last 184.78) total: 4500 /unlink in 23.17 seconds: 194.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1068 (time 1778927766.68 total 10.01 last 106.71) - open/close 2202 (time 1778927776.69 total 20.01 last 113.35) - open/close 3219 (time 1778927786.70 total 30.03 last 101.52) - open/close 4302 (time 1778927796.71 total 40.04 last 108.25) total: 4500 open/close in 41.81 seconds: 107.62 ops/second - /unlink 2142 (time 1778927811.20 total 10.00 last 214.20) - /unlink 4054 (time 1778927821.21 total 20.01 last 191.11) total: 4500 /unlink in 22.42 seconds: 200.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 956 (time 1778927836.52 total 10.01 last 95.54) - open/close 2139 (time 1778927846.52 total 20.01 last 118.29) - open/close 3132 (time 1778927856.52 total 30.01 last 99.28) - open/close 4216 (time 1778927866.52 total 40.01 last 108.35) total: 4500 open/close in 42.48 seconds: 105.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1892 (time 1778927882.01 total 10.00 last 189.20) - /unlink 3932 (time 1778927892.01 total 20.00 last 203.91) total: 4500 /unlink in 22.92 seconds: 196.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1184 (time 1778927907.31 total 10.00 last 118.38) - open/close 2490 (time 1778927917.31 total 20.00 last 130.57) - open/close 3754 (time 1778927927.32 total 30.01 last 126.35) total: 4500 open/close in 36.46 seconds: 123.43 ops/second - /unlink 2320 (time 1778927946.26 total 10.00 last 231.98) - /unlink 4434 (time 1778927956.26 total 20.00 last 211.35) total: 4500 /unlink in 20.34 seconds: 221.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 914 (time 1778927969.88 total 10.00 last 91.37) - open/close 1864 (time 1778927979.89 total 20.01 last 94.95) - open/close 2832 (time 1778927989.90 total 30.02 last 96.72) - open/close 3727 (time 1778927999.90 total 40.02 last 89.47) total: 4500 open/close in 47.72 seconds: 94.31 ops/second - /unlink 1711 (time 1778928020.47 total 10.00 last 171.10) - /unlink 3539 (time 1778928030.48 total 20.00 last 182.78) total: 4500 /unlink in 26.17 seconds: 171.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 872 (time 1778928049.70 total 10.02 last 86.98) - open/close 1924 (time 1778928059.71 total 20.03 last 105.18) - open/close 2911 (time 1778928069.72 total 30.04 last 98.61) - open/close 3795 (time 1778928079.72 total 40.04 last 88.39) total: 4500 open/close in 47.62 seconds: 94.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1830 (time 1778928100.27 total 10.00 last 182.99) - /unlink 3503 (time 1778928110.27 total 20.00 last 167.25) total: 4500 /unlink in 25.79 seconds: 174.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 930 (time 1778928129.06 total 10.01 last 92.93) - open/close 2092 (time 1778928139.06 total 20.01 last 116.13) - open/close 3147 (time 1778928149.07 total 30.02 last 105.42) - open/close 4176 (time 1778928159.07 total 40.02 last 102.90) total: 4500 open/close in 42.98 seconds: 104.70 ops/second - /unlink 1898 (time 1778928174.85 total 10.00 last 189.74) - /unlink 3854 (time 1778928184.85 total 20.00 last 195.57) total: 4500 /unlink in 23.42 seconds: 192.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1181 (time 1778928201.08 total 10.01 last 117.96) - open/close 2110 (time 1778928211.09 total 20.02 last 92.85) - open/close 3167 (time 1778928221.10 total 30.03 last 105.57) - open/close 4223 (time 1778928231.11 total 40.04 last 105.54) total: 4500 open/close in 42.51 seconds: 105.86 ops/second - /unlink 2175 (time 1778928246.18 total 10.00 last 217.47) - /unlink 4195 (time 1778928256.19 total 20.01 last 201.81) total: 4500 /unlink in 21.38 seconds: 210.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1020 (time 1778928270.52 total 10.01 last 101.92) - open/close 2183 (time 1778928280.53 total 20.01 last 116.28) - open/close 3391 (time 1778928290.54 total 30.02 last 120.62) - open/close 4468 (time 1778928300.55 total 40.03 last 107.62) total: 4500 open/close in 40.33 seconds: 111.58 ops/second - /unlink 2165 (time 1778928313.42 total 10.00 last 216.41) - /unlink 4433 (time 1778928323.42 total 20.01 last 226.76) total: 4500 /unlink in 20.38 seconds: 220.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1202 (time 1778928336.18 total 10.01 last 120.12) - open/close 2363 (time 1778928346.18 total 20.01 last 116.09) - open/close 3757 (time 1778928356.18 total 30.01 last 139.38) total: 4500 open/close in 34.53 seconds: 130.32 ops/second - /unlink 3076 (time 1778928372.90 total 10.00 last 307.57) total: 4500 /unlink in 14.90 seconds: 301.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1468 (time 1778928389.65 total 10.01 last 146.72) - open/close 3030 (time 1778928399.65 total 20.01 last 156.15) total: 4500 open/close in 28.53 seconds: 157.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3079 (time 1778928420.03 total 10.00 last 307.88) total: 4500 /unlink in 14.74 seconds: 305.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1431 (time 1778928436.59 total 10.01 last 143.01) - open/close 3014 (time 1778928446.59 total 20.01 last 158.26) total: 4500 open/close in 29.60 seconds: 152.05 ops/second - /unlink 3020 (time 1778928467.97 total 10.00 last 301.97) total: 4500 /unlink in 13.61 seconds: 330.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1643 (time 1778928483.46 total 10.00 last 164.22) - open/close 3075 (time 1778928493.46 total 20.01 last 143.19) total: 4500 open/close in 29.27 seconds: 153.73 ops/second - /unlink 4001 (time 1778928514.64 total 10.00 last 400.06) total: 4500 /unlink in 11.03 seconds: 407.86 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11196: 94593 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg246-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg246-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg246-server stop mds service on oleg246-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg246-server stop mds service on oleg246-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg246-server unloading modules via unload_modules_local on: 'oleg246-server' oleg246-server: oleg246-server.virtnet: executing unload_modules_local oleg246-server: modules unloaded.