== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 06:18:54 (1787480334) Stopping clients: oleg232-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg232-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg232-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg232-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg232-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg232-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg232-server oleg232-server: oleg232-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg232-server' oleg232-server: oleg232-server.virtnet: executing load_modules_local oleg232-server: Loading modules from /home/green/git/lustre-release/lustre oleg232-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg232-server: detected 4 online CPUs by sysfs oleg232-server: MODOPTS_LIBCFS= oleg232-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg232-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg232-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg232-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg232-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg232-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg232-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg232-client.virtnet: -o user_xattr,flock 192.168.202.132@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 795 (time 1787480547.22 total 10.00 last 79.49) - open/close 1655 (time 1787480557.22 total 20.00 last 85.98) - open/close 2558 (time 1787480567.22 total 30.01 last 90.25) - open/close 3239 (time 1787480577.23 total 40.01 last 68.08) - open/close 4036 (time 1787480587.23 total 50.01 last 79.68) total: 4500 open/close in 55.17 seconds: 81.57 ops/second - /unlink 1613 (time 1787480606.91 total 10.00 last 161.23) - /unlink 3351 (time 1787480616.91 total 20.01 last 173.76) total: 4500 /unlink in 26.49 seconds: 169.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 839 (time 1787480637.69 total 10.00 last 83.88) - open/close 1665 (time 1787480647.69 total 20.01 last 82.57) - open/close 2694 (time 1787480657.70 total 30.02 last 102.76) - open/close 3713 (time 1787480667.71 total 40.03 last 101.82) total: 4500 open/close in 49.36 seconds: 91.17 ops/second - /unlink 1563 (time 1787480691.62 total 10.00 last 156.27) - /unlink 3189 (time 1787480701.63 total 20.01 last 162.52) total: 4500 /unlink in 29.30 seconds: 153.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 969 (time 1787480726.64 total 10.01 last 96.81) - open/close 1853 (time 1787480736.65 total 20.01 last 88.35) - open/close 2827 (time 1787480746.65 total 30.02 last 97.35) - open/close 3752 (time 1787480756.66 total 40.02 last 92.48) total: 4500 open/close in 48.09 seconds: 93.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1518 (time 1787480778.89 total 10.00 last 151.77) - /unlink 3050 (time 1787480788.89 total 20.00 last 153.19) total: 4500 /unlink in 29.04 seconds: 154.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 825 (time 1787480812.70 total 10.00 last 82.49) - open/close 1650 (time 1787480822.71 total 20.01 last 82.46) - open/close 2594 (time 1787480832.72 total 30.02 last 94.29) - open/close 3463 (time 1787480842.74 total 40.03 last 86.74) - open/close 4359 (time 1787480852.78 total 50.08 last 89.22) total: 4500 open/close in 52.16 seconds: 86.27 ops/second - /unlink 1495 (time 1787480869.14 total 10.01 last 149.37) - /unlink 3297 (time 1787480879.14 total 20.01 last 180.18) total: 4500 /unlink in 27.28 seconds: 164.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 945 (time 1787480900.31 total 10.04 last 94.15) - open/close 1855 (time 1787480910.34 total 20.07 last 90.71) - open/close 2692 (time 1787480920.34 total 30.08 last 83.64) - open/close 3694 (time 1787480930.35 total 40.08 last 100.12) total: 4500 open/close in 48.69 seconds: 92.42 ops/second - /unlink 1590 (time 1787480952.80 total 10.00 last 158.93) - /unlink 3231 (time 1787480962.80 total 20.01 last 164.04) total: 4500 /unlink in 27.07 seconds: 166.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 901 (time 1787480983.59 total 10.00 last 90.09) - open/close 1809 (time 1787480993.60 total 20.01 last 90.73) - open/close 2636 (time 1787481003.60 total 30.02 last 82.64) - open/close 3459 (time 1787481013.61 total 40.02 last 82.29) - open/close 4355 (time 1787481023.61 total 50.02 last 89.54) total: 4500 open/close in 51.36 seconds: 87.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1767 (time 1787481039.13 total 10.00 last 176.66) - /unlink 3525 (time 1787481049.14 total 20.01 last 175.68) total: 4500 /unlink in 25.44 seconds: 176.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1024 (time 1787481068.09 total 10.00 last 102.35) - open/close 1913 (time 1787481078.10 total 20.01 last 88.87) - open/close 2833 (time 1787481088.10 total 30.02 last 91.94) - open/close 3583 (time 1787481098.13 total 40.04 last 74.81) - open/close 4490 (time 1787481108.13 total 50.04 last 90.68) total: 4500 open/close in 50.18 seconds: 89.68 ops/second - /unlink 1439 (time 1787481123.49 total 10.00 last 143.88) - /unlink 2866 (time 1787481133.49 total 20.01 last 142.59) - /unlink 4296 (time 1787481143.50 total 30.02 last 142.90) total: 4500 /unlink in 31.71 seconds: 141.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 820 (time 1787481160.34 total 10.01 last 81.93) - open/close 1644 (time 1787481170.34 total 20.01 last 82.38) - open/close 2504 (time 1787481180.35 total 30.01 last 85.97) - open/close 3404 (time 1787481190.35 total 40.02 last 89.96) - open/close 4297 (time 1787481200.35 total 50.02 last 89.30) total: 4500 open/close in 52.69 seconds: 85.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1653 (time 1787481217.50 total 10.00 last 165.27) - /unlink 3360 (time 1787481227.51 total 20.02 last 170.39) total: 4500 /unlink in 26.63 seconds: 168.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 848 (time 1787481249.23 total 10.00 last 84.79) - open/close 1710 (time 1787481259.24 total 20.01 last 86.15) - open/close 2594 (time 1787481269.24 total 30.01 last 88.37) - open/close 3592 (time 1787481279.25 total 40.01 last 99.77) total: 4500 open/close in 49.61 seconds: 90.70 ops/second - /unlink 1587 (time 1787481302.94 total 10.00 last 158.64) - /unlink 3191 (time 1787481312.94 total 20.00 last 160.39) total: 4500 /unlink in 27.96 seconds: 160.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 906 (time 1787481335.17 total 10.00 last 90.57) - open/close 1572 (time 1787481345.17 total 20.01 last 66.59) - open/close 2373 (time 1787481355.17 total 30.01 last 80.10) - open/close 3167 (time 1787481365.17 total 40.01 last 79.38) - open/close 4073 (time 1787481375.18 total 50.01 last 90.55) total: 4500 open/close in 56.68 seconds: 79.39 ops/second - /unlink 1359 (time 1787481398.74 total 10.01 last 135.80) - /unlink 2967 (time 1787481408.74 total 20.01 last 160.80) - /unlink 4344 (time 1787481418.74 total 30.02 last 137.58) total: 4500 /unlink in 31.42 seconds: 143.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 799 (time 1787481435.29 total 10.00 last 79.86) - open/close 1595 (time 1787481445.29 total 20.01 last 79.55) - open/close 2389 (time 1787481455.29 total 30.01 last 79.39) - open/close 3362 (time 1787481465.30 total 40.01 last 97.27) - open/close 4298 (time 1787481475.32 total 50.03 last 93.42) total: 4500 open/close in 52.28 seconds: 86.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1606 (time 1787481492.21 total 10.05 last 159.80) - /unlink 3214 (time 1787481502.21 total 20.05 last 160.80) total: 4500 /unlink in 27.42 seconds: 164.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 851 (time 1787481524.81 total 10.01 last 85.05) - open/close 1537 (time 1787481534.82 total 20.01 last 68.57) - open/close 2379 (time 1787481544.83 total 30.02 last 84.13) - open/close 3242 (time 1787481554.85 total 40.04 last 86.10) - open/close 4111 (time 1787481564.85 total 50.04 last 86.89) total: 4500 open/close in 54.47 seconds: 82.61 ops/second - /unlink 1943 (time 1787481583.62 total 10.00 last 194.25) - /unlink 3875 (time 1787481593.63 total 20.01 last 192.97) total: 4500 /unlink in 23.91 seconds: 188.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1086 (time 1787481611.90 total 10.01 last 108.53) - open/close 2069 (time 1787481621.91 total 20.01 last 98.26) - open/close 2938 (time 1787481631.92 total 30.02 last 86.80) - open/close 3893 (time 1787481641.92 total 40.03 last 95.44) total: 4500 open/close in 46.69 seconds: 96.39 ops/second - /unlink 1538 (time 1787481661.94 total 10.00 last 153.76) - /unlink 3153 (time 1787481671.94 total 20.00 last 161.50) total: 4500 /unlink in 27.71 seconds: 162.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1029 (time 1787481693.22 total 10.01 last 102.83) - open/close 1976 (time 1787481703.23 total 20.02 last 94.62) - open/close 2962 (time 1787481713.23 total 30.02 last 98.54) - open/close 3942 (time 1787481723.24 total 40.03 last 97.92) total: 4500 open/close in 46.05 seconds: 97.72 ops/second - /unlink 1500 (time 1787481743.70 total 10.00 last 149.94) - /unlink 3275 (time 1787481753.70 total 20.01 last 177.42) total: 4500 /unlink in 27.37 seconds: 164.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 823 (time 1787481774.45 total 10.00 last 82.27) - open/close 1641 (time 1787481784.46 total 20.01 last 81.77) - open/close 2471 (time 1787481794.47 total 30.02 last 82.89) - open/close 3397 (time 1787481804.47 total 40.03 last 92.55) - open/close 3935 (time 1787481814.51 total 50.06 last 53.62) total: 4500 open/close in 57.67 seconds: 78.03 ops/second - /unlink 1836 (time 1787481837.68 total 10.01 last 183.33) - /unlink 3487 (time 1787481847.68 total 20.02 last 165.03) total: 4500 /unlink in 26.78 seconds: 168.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 910 (time 1787481869.27 total 10.01 last 90.90) - open/close 1678 (time 1787481879.28 total 20.02 last 76.73) - open/close 2540 (time 1787481889.28 total 30.03 last 86.15) - open/close 3405 (time 1787481899.29 total 40.03 last 86.43) - open/close 4254 (time 1787481909.30 total 50.04 last 84.84) total: 4500 open/close in 52.54 seconds: 85.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1734 (time 1787481925.97 total 10.01 last 173.25) - /unlink 3203 (time 1787481935.98 total 20.01 last 146.86) total: 4500 /unlink in 28.04 seconds: 160.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 868 (time 1787481959.03 total 10.00 last 86.76) - open/close 1695 (time 1787481969.04 total 20.02 last 82.61) - open/close 2501 (time 1787481979.08 total 30.05 last 80.32) - open/close 3335 (time 1787481989.08 total 40.06 last 83.35) - open/close 4113 (time 1787481999.09 total 50.06 last 77.80) total: 4500 open/close in 54.37 seconds: 82.77 ops/second - /unlink 1592 (time 1787482018.08 total 10.00 last 159.20) - /unlink 3470 (time 1787482028.08 total 20.00 last 187.76) total: 4500 /unlink in 27.25 seconds: 165.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 601 (time 1787482050.51 total 10.00 last 60.09) - open/close 1319 (time 1787482060.51 total 20.01 last 71.78) - open/close 2144 (time 1787482070.51 total 30.01 last 82.49) - open/close 2989 (time 1787482080.52 total 40.01 last 84.47) - open/close 3799 (time 1787482090.53 total 50.02 last 80.91) total: 4500 open/close in 59.01 seconds: 76.26 ops/second - /unlink 1802 (time 1787482113.73 total 10.00 last 180.20) - /unlink 3470 (time 1787482123.74 total 20.01 last 166.65) total: 4500 /unlink in 26.91 seconds: 167.22 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95184 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg232-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg232-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg232-server stop mds service on oleg232-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg232-server stop mds service on oleg232-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg232-server unloading modules via unload_modules_local on: 'oleg232-server' oleg232-server: oleg232-server.virtnet: executing unload_modules_local oleg232-server: modules unloaded.