== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 01:03:46 (1788843826) Stopping clients: oleg646-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg646-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg646-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg646-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg646-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg646-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg646-server oleg646-server: oleg646-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg646-server' oleg646-server: oleg646-server.virtnet: executing load_modules_local oleg646-server: kmemleak disabled oleg646-server: Loading modules from /home/green/git/lustre-release/lustre oleg646-server: detected 4 online CPUs by sysfs oleg646-server: MODOPTS_LIBCFS= oleg646-server: Force libcfs to create 2 CPU partitions oleg646-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg646-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg646-server: oleg646-server.virtnet: executing set_default_debug -1 all pdsh@oleg646-client: oleg646-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg646-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg646-server: oleg646-server.virtnet: executing set_default_debug -1 all pdsh@oleg646-client: oleg646-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg646-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg646-server: oleg646-server.virtnet: executing set_default_debug -1 all pdsh@oleg646-client: oleg646-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg646-client.virtnet: -o user_xattr,flock 192.168.206.146@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 851 (time 1788844000.98 total 10.01 last 85.00) - open/close 1718 (time 1788844010.98 total 20.02 last 86.64) - open/close 2554 (time 1788844021.00 total 30.03 last 83.49) - open/close 3425 (time 1788844031.00 total 40.03 last 87.08) - open/close 4282 (time 1788844041.00 total 50.04 last 85.68) total: 4500 open/close in 52.57 seconds: 85.60 ops/second - /unlink 1665 (time 1788844057.41 total 10.00 last 166.45) - /unlink 3206 (time 1788844067.42 total 20.01 last 154.03) total: 4500 /unlink in 29.34 seconds: 153.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 904 (time 1788844091.06 total 10.01 last 90.32) - open/close 1820 (time 1788844101.06 total 20.01 last 91.60) - open/close 2743 (time 1788844111.06 total 30.01 last 92.29) - open/close 3641 (time 1788844121.06 total 40.01 last 89.78) total: 4500 open/close in 49.53 seconds: 90.86 ops/second - /unlink 1714 (time 1788844144.37 total 10.00 last 171.39) - /unlink 3340 (time 1788844154.38 total 20.00 last 162.57) total: 4500 /unlink in 27.31 seconds: 164.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 911 (time 1788844175.66 total 10.01 last 91.00) - open/close 1831 (time 1788844185.66 total 20.01 last 91.97) - open/close 2740 (time 1788844195.67 total 30.02 last 90.86) - open/close 3589 (time 1788844205.67 total 40.02 last 84.85) - open/close 4466 (time 1788844215.68 total 50.03 last 87.62) total: 4500 open/close in 50.45 seconds: 89.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1734 (time 1788844229.55 total 10.00 last 173.36) - /unlink 3397 (time 1788844239.55 total 20.01 last 166.21) total: 4500 /unlink in 26.16 seconds: 172.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 833 (time 1788844259.95 total 10.02 last 83.11) - open/close 1732 (time 1788844269.96 total 20.04 last 89.79) - open/close 2541 (time 1788844279.97 total 30.04 last 80.86) - open/close 3452 (time 1788844289.98 total 40.05 last 91.00) - open/close 4355 (time 1788844299.98 total 50.05 last 90.27) total: 4500 open/close in 51.42 seconds: 87.51 ops/second - /unlink 1590 (time 1788844315.88 total 10.00 last 158.94) - /unlink 3114 (time 1788844325.88 total 20.00 last 152.40) total: 4500 /unlink in 28.63 seconds: 157.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 862 (time 1788844348.69 total 10.01 last 86.12) - open/close 1714 (time 1788844358.71 total 20.03 last 85.04) - open/close 2642 (time 1788844368.71 total 30.03 last 92.73) - open/close 3530 (time 1788844378.72 total 40.04 last 88.79) - open/close 4423 (time 1788844388.73 total 50.05 last 89.21) total: 4500 open/close in 51.02 seconds: 88.20 ops/second - /unlink 1718 (time 1788844403.55 total 10.01 last 171.70) - /unlink 3269 (time 1788844413.57 total 20.02 last 154.82) total: 4500 /unlink in 27.15 seconds: 165.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 734 (time 1788844435.22 total 10.02 last 73.28) - open/close 1586 (time 1788844445.22 total 20.02 last 85.19) - open/close 2449 (time 1788844455.23 total 30.03 last 86.19) - open/close 3292 (time 1788844465.24 total 40.03 last 84.26) - open/close 4186 (time 1788844475.24 total 50.04 last 89.35) total: 4500 open/close in 54.68 seconds: 82.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1670 (time 1788844494.43 total 10.00 last 166.98) - /unlink 3368 (time 1788844504.43 total 20.00 last 169.79) total: 4500 /unlink in 26.76 seconds: 168.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 689 (time 1788844526.26 total 10.01 last 68.85) - open/close 1626 (time 1788844536.27 total 20.02 last 93.61) - open/close 2575 (time 1788844546.27 total 30.02 last 94.89) - open/close 3463 (time 1788844556.28 total 40.03 last 88.73) - open/close 4215 (time 1788844566.28 total 50.03 last 75.19) total: 4500 open/close in 54.02 seconds: 83.30 ops/second - /unlink 1608 (time 1788844585.46 total 10.00 last 160.76) - /unlink 3335 (time 1788844595.47 total 20.01 last 172.65) total: 4500 /unlink in 26.97 seconds: 166.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 880 (time 1788844616.56 total 10.00 last 87.98) - open/close 1532 (time 1788844626.60 total 20.05 last 64.89) - open/close 2446 (time 1788844636.61 total 30.05 last 91.37) - open/close 3203 (time 1788844646.62 total 40.06 last 75.64) - open/close 4017 (time 1788844656.64 total 50.08 last 81.22) total: 4500 open/close in 55.92 seconds: 80.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1669 (time 1788844676.38 total 10.00 last 166.86) - /unlink 3425 (time 1788844686.39 total 20.01 last 175.45) total: 4500 /unlink in 26.77 seconds: 168.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 825 (time 1788844707.39 total 10.00 last 82.49) - open/close 1693 (time 1788844717.39 total 20.00 last 86.80) - open/close 2592 (time 1788844727.40 total 30.01 last 89.82) - open/close 3458 (time 1788844737.40 total 40.01 last 86.59) - open/close 4236 (time 1788844747.40 total 50.01 last 77.80) total: 4500 open/close in 53.00 seconds: 84.91 ops/second - /unlink 1644 (time 1788844764.62 total 10.00 last 164.38) - /unlink 3210 (time 1788844774.62 total 20.00 last 156.57) total: 4500 /unlink in 28.44 seconds: 158.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 869 (time 1788844797.44 total 10.01 last 86.83) - open/close 1754 (time 1788844807.47 total 20.03 last 88.28) - open/close 2681 (time 1788844817.47 total 30.04 last 92.64) - open/close 3515 (time 1788844827.49 total 40.05 last 83.29) - open/close 4390 (time 1788844837.49 total 50.06 last 87.44) total: 4500 open/close in 51.34 seconds: 87.64 ops/second - /unlink 1771 (time 1788844852.31 total 10.00 last 177.03) - /unlink 3443 (time 1788844862.32 total 20.01 last 167.14) total: 4500 /unlink in 26.60 seconds: 169.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 890 (time 1788844882.50 total 10.00 last 88.98) - open/close 1820 (time 1788844892.51 total 20.01 last 92.92) - open/close 2683 (time 1788844902.51 total 30.01 last 86.28) - open/close 3524 (time 1788844912.51 total 40.02 last 84.07) - open/close 4472 (time 1788844922.52 total 50.03 last 94.72) total: 4500 open/close in 50.31 seconds: 89.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1771 (time 1788844936.98 total 10.00 last 177.10) - /unlink 3308 (time 1788844946.99 total 20.00 last 153.64) total: 4500 /unlink in 27.32 seconds: 164.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 822 (time 1788844968.48 total 10.00 last 82.19) - open/close 1572 (time 1788844978.49 total 20.01 last 74.95) - open/close 2497 (time 1788844988.49 total 30.01 last 92.49) - open/close 3361 (time 1788844998.49 total 40.01 last 86.40) - open/close 4246 (time 1788845008.49 total 50.01 last 88.49) total: 4500 open/close in 52.77 seconds: 85.27 ops/second - /unlink 1635 (time 1788845025.33 total 10.01 last 163.39) - /unlink 3359 (time 1788845035.33 total 20.01 last 172.34) total: 4500 /unlink in 26.75 seconds: 168.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 872 (time 1788845055.64 total 10.00 last 87.16) - open/close 1692 (time 1788845065.65 total 20.01 last 81.96) - open/close 2486 (time 1788845075.66 total 30.02 last 79.31) - open/close 3276 (time 1788845085.66 total 40.02 last 78.98) - open/close 4178 (time 1788845095.67 total 50.04 last 90.09) total: 4500 open/close in 53.52 seconds: 84.08 ops/second - /unlink 1709 (time 1788845113.15 total 10.02 last 170.58) - /unlink 3415 (time 1788845123.15 total 20.02 last 170.54) total: 4500 /unlink in 26.49 seconds: 169.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 854 (time 1788845143.68 total 10.00 last 85.37) - open/close 1803 (time 1788845153.69 total 20.01 last 94.84) - open/close 2753 (time 1788845163.69 total 30.01 last 94.96) - open/close 3667 (time 1788845173.71 total 40.03 last 91.27) total: 4500 open/close in 49.21 seconds: 91.45 ops/second - /unlink 1752 (time 1788845196.99 total 10.00 last 175.17) - /unlink 3392 (time 1788845206.99 total 20.00 last 163.95) total: 4500 /unlink in 27.56 seconds: 163.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 895 (time 1788845229.77 total 10.00 last 89.47) - open/close 1643 (time 1788845239.77 total 20.01 last 74.75) - open/close 2489 (time 1788845249.77 total 30.01 last 84.59) - open/close 3361 (time 1788845259.77 total 40.01 last 87.19) - open/close 4115 (time 1788845269.78 total 50.02 last 75.36) total: 4500 open/close in 54.25 seconds: 82.95 ops/second - /unlink 1619 (time 1788845288.44 total 10.00 last 161.87) - /unlink 3258 (time 1788845298.45 total 20.01 last 163.82) total: 4500 /unlink in 29.20 seconds: 154.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 779 (time 1788845321.78 total 10.01 last 77.85) - open/close 1518 (time 1788845331.79 total 20.01 last 73.86) - open/close 2321 (time 1788845341.79 total 30.01 last 80.29) - open/close 3138 (time 1788845351.79 total 40.02 last 81.68) - open/close 4047 (time 1788845361.79 total 50.02 last 90.89) total: 4500 open/close in 54.75 seconds: 82.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1557 (time 1788845380.33 total 10.00 last 155.66) - /unlink 3264 (time 1788845390.34 total 20.01 last 170.64) total: 4500 /unlink in 27.09 seconds: 166.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 933 (time 1788845411.35 total 10.01 last 93.23) - open/close 1877 (time 1788845421.35 total 20.01 last 94.34) - open/close 2822 (time 1788845431.36 total 30.02 last 94.42) - open/close 3736 (time 1788845441.37 total 40.03 last 91.36) total: 4500 open/close in 48.84 seconds: 92.14 ops/second - /unlink 1721 (time 1788845463.51 total 10.01 last 171.94) - /unlink 3451 (time 1788845473.52 total 20.02 last 172.85) total: 4500 /unlink in 25.98 seconds: 173.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 897 (time 1788845492.91 total 10.00 last 89.67) - open/close 1829 (time 1788845502.91 total 20.00 last 93.19) - open/close 2723 (time 1788845512.91 total 30.01 last 89.35) - open/close 3604 (time 1788845522.92 total 40.01 last 88.06) - open/close 4455 (time 1788845532.92 total 50.02 last 85.06) total: 4500 open/close in 50.47 seconds: 89.16 ops/second - /unlink 1632 (time 1788845547.29 total 10.00 last 163.17) - /unlink 3289 (time 1788845557.29 total 20.00 last 165.67) total: 4500 /unlink in 27.24 seconds: 165.22 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95039 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg646-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg646-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg646-server stop mds service on oleg646-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg646-server stop mds service on oleg646-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg646-server unloading modules via unload_modules_local on: 'oleg646-server' oleg646-server: oleg646-server.virtnet: executing unload_modules_local oleg646-server: kmemleak disabled modules unloaded.