== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 12:16:26 (1788624986) Stopping clients: oleg608-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg608-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg608-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg608-server oleg608-server: oleg608-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing load_modules_local oleg608-server: kmemleak disabled oleg608-server: Loading modules from /home/green/git/lustre-release/lustre oleg608-server: detected 4 online CPUs by sysfs oleg608-server: MODOPTS_LIBCFS= oleg608-server: Force libcfs to create 2 CPU partitions oleg608-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg608-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg608-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg608-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg608-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg608-client.virtnet: -o user_xattr,flock 192.168.206.108@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 847 (time 1788625178.17 total 10.00 last 84.66) - open/close 1670 (time 1788625188.17 total 20.01 last 82.26) - open/close 2545 (time 1788625198.18 total 30.01 last 87.47) - open/close 3437 (time 1788625208.18 total 40.02 last 89.14) - open/close 4320 (time 1788625218.19 total 50.02 last 88.25) total: 4500 open/close in 52.03 seconds: 86.49 ops/second - /unlink 1674 (time 1788625233.99 total 10.01 last 167.25) - /unlink 3219 (time 1788625243.99 total 20.01 last 154.49) total: 4500 /unlink in 27.66 seconds: 162.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 893 (time 1788625266.33 total 10.01 last 89.25) - open/close 1882 (time 1788625276.33 total 20.01 last 98.89) - open/close 2746 (time 1788625286.33 total 30.01 last 86.37) - open/close 3602 (time 1788625296.33 total 40.01 last 85.56) total: 4500 open/close in 49.66 seconds: 90.62 ops/second - /unlink 1659 (time 1788625320.07 total 10.00 last 165.83) - /unlink 3299 (time 1788625330.07 total 20.01 last 163.93) total: 4500 /unlink in 27.60 seconds: 163.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 886 (time 1788625352.42 total 10.00 last 88.58) - open/close 1808 (time 1788625362.42 total 20.00 last 92.18) - open/close 2726 (time 1788625372.42 total 30.01 last 91.79) - open/close 3593 (time 1788625382.42 total 40.01 last 86.67) - open/close 4487 (time 1788625392.43 total 50.02 last 89.34) total: 4500 open/close in 50.11 seconds: 89.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1655 (time 1788625406.85 total 10.00 last 165.49) - /unlink 3301 (time 1788625416.86 total 20.01 last 164.47) total: 4500 /unlink in 27.39 seconds: 164.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 894 (time 1788625437.77 total 10.01 last 89.35) - open/close 1856 (time 1788625447.77 total 20.01 last 96.16) - open/close 2749 (time 1788625457.79 total 30.02 last 89.18) - open/close 3599 (time 1788625467.79 total 40.03 last 84.94) - open/close 4492 (time 1788625477.80 total 50.04 last 89.24) total: 4500 open/close in 50.09 seconds: 89.85 ops/second - /unlink 1608 (time 1788625491.56 total 10.00 last 160.79) - /unlink 3185 (time 1788625501.57 total 20.01 last 157.56) total: 4500 /unlink in 28.49 seconds: 157.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 828 (time 1788625524.39 total 10.01 last 82.68) - open/close 1689 (time 1788625534.40 total 20.02 last 86.05) - open/close 2606 (time 1788625544.40 total 30.03 last 91.66) - open/close 3392 (time 1788625554.40 total 40.03 last 78.57) - open/close 4246 (time 1788625564.42 total 50.04 last 85.29) total: 4500 open/close in 52.73 seconds: 85.34 ops/second - /unlink 1597 (time 1788625581.32 total 10.00 last 159.64) - /unlink 3207 (time 1788625591.33 total 20.01 last 160.90) total: 4500 /unlink in 28.09 seconds: 160.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 905 (time 1788625613.15 total 10.01 last 90.41) - open/close 1737 (time 1788625623.16 total 20.02 last 83.13) - open/close 2629 (time 1788625633.16 total 30.02 last 89.18) - open/close 3569 (time 1788625643.18 total 40.03 last 93.87) - open/close 4427 (time 1788625653.21 total 50.07 last 85.53) total: 4500 open/close in 50.83 seconds: 88.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1597 (time 1788625667.95 total 10.00 last 159.65) - /unlink 3269 (time 1788625677.96 total 20.01 last 167.09) total: 4500 /unlink in 27.68 seconds: 162.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 917 (time 1788625700.02 total 10.00 last 91.66) - open/close 1809 (time 1788625710.02 total 20.01 last 89.14) - open/close 2723 (time 1788625720.04 total 30.03 last 91.23) - open/close 3580 (time 1788625730.06 total 40.05 last 85.56) - open/close 4492 (time 1788625740.06 total 50.05 last 91.17) total: 4500 open/close in 50.14 seconds: 89.75 ops/second - /unlink 1655 (time 1788625753.83 total 10.00 last 165.44) - /unlink 3233 (time 1788625763.83 total 20.01 last 157.73) total: 4500 /unlink in 28.21 seconds: 159.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 861 (time 1788625786.44 total 10.01 last 85.98) - open/close 1752 (time 1788625796.44 total 20.02 last 89.08) - open/close 2731 (time 1788625806.44 total 30.02 last 97.88) - open/close 3616 (time 1788625816.45 total 40.02 last 88.45) total: 4500 open/close in 49.93 seconds: 90.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1687 (time 1788625839.75 total 10.01 last 168.62) - /unlink 3388 (time 1788625849.75 total 20.01 last 170.09) total: 4500 /unlink in 26.39 seconds: 170.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 858 (time 1788625869.71 total 10.01 last 85.76) - open/close 1738 (time 1788625879.72 total 20.01 last 87.97) - open/close 2565 (time 1788625889.73 total 30.02 last 82.58) - open/close 3459 (time 1788625899.73 total 40.02 last 89.39) - open/close 4312 (time 1788625909.74 total 50.03 last 85.26) total: 4500 open/close in 52.14 seconds: 86.30 ops/second - /unlink 1711 (time 1788625925.51 total 10.00 last 171.07) - /unlink 3378 (time 1788625935.51 total 20.01 last 166.64) total: 4500 /unlink in 26.70 seconds: 168.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 873 (time 1788625955.98 total 10.01 last 87.23) - open/close 1724 (time 1788625965.98 total 20.01 last 85.09) - open/close 2641 (time 1788625975.99 total 30.01 last 91.65) - open/close 3493 (time 1788625985.99 total 40.02 last 85.16) - open/close 4344 (time 1788625996.00 total 50.03 last 85.01) total: 4500 open/close in 51.68 seconds: 87.08 ops/second - /unlink 1708 (time 1788626011.78 total 10.00 last 170.75) - /unlink 3504 (time 1788626021.78 total 20.01 last 179.52) total: 4500 /unlink in 25.91 seconds: 173.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 912 (time 1788626040.83 total 10.00 last 91.17) - open/close 1880 (time 1788626050.84 total 20.01 last 96.70) - open/close 2859 (time 1788626060.85 total 30.03 last 97.78) - open/close 3685 (time 1788626070.86 total 40.04 last 82.51) total: 4500 open/close in 48.82 seconds: 92.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1753 (time 1788626093.14 total 10.00 last 175.25) - /unlink 3378 (time 1788626103.14 total 20.00 last 162.49) total: 4500 /unlink in 27.30 seconds: 164.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 918 (time 1788626125.23 total 10.01 last 91.73) - open/close 1836 (time 1788626135.24 total 20.01 last 91.76) - open/close 2756 (time 1788626145.24 total 30.02 last 91.96) - open/close 3641 (time 1788626155.24 total 40.02 last 88.50) total: 4500 open/close in 49.19 seconds: 91.49 ops/second - /unlink 1735 (time 1788626177.82 total 10.01 last 173.38) - /unlink 3371 (time 1788626187.82 total 20.01 last 163.59) total: 4500 /unlink in 26.87 seconds: 167.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 836 (time 1788626208.36 total 10.00 last 83.58) - open/close 1691 (time 1788626218.37 total 20.01 last 85.46) - open/close 2576 (time 1788626228.37 total 30.01 last 88.45) - open/close 3510 (time 1788626238.37 total 40.01 last 93.38) - open/close 4456 (time 1788626248.38 total 50.02 last 94.57) total: 4500 open/close in 50.43 seconds: 89.23 ops/second - /unlink 1769 (time 1788626262.21 total 10.00 last 176.85) - /unlink 3436 (time 1788626272.21 total 20.01 last 166.64) total: 4500 /unlink in 26.89 seconds: 167.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 888 (time 1788626293.76 total 10.02 last 88.62) - open/close 1800 (time 1788626303.77 total 20.02 last 91.19) - open/close 2606 (time 1788626313.78 total 30.04 last 80.47) - open/close 3524 (time 1788626323.79 total 40.04 last 91.74) - open/close 4349 (time 1788626333.79 total 50.05 last 82.48) total: 4500 open/close in 52.17 seconds: 86.25 ops/second - /unlink 1694 (time 1788626349.47 total 10.00 last 169.32) - /unlink 3374 (time 1788626359.47 total 20.01 last 167.98) total: 4500 /unlink in 26.52 seconds: 169.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 967 (time 1788626378.91 total 10.00 last 96.66) - open/close 1918 (time 1788626388.91 total 20.00 last 95.09) - open/close 2887 (time 1788626398.92 total 30.01 last 96.85) - open/close 3799 (time 1788626408.92 total 40.01 last 91.18) total: 4500 open/close in 47.91 seconds: 93.93 ops/second - /unlink 1749 (time 1788626430.59 total 10.00 last 174.89) - /unlink 3489 (time 1788626440.60 total 20.00 last 173.96) total: 4500 /unlink in 26.44 seconds: 170.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 925 (time 1788626461.00 total 10.00 last 92.46) - open/close 1773 (time 1788626471.01 total 20.01 last 84.75) - open/close 2715 (time 1788626481.02 total 30.02 last 94.11) - open/close 3678 (time 1788626491.02 total 40.02 last 96.27) total: 4500 open/close in 49.27 seconds: 91.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1648 (time 1788626513.18 total 10.02 last 164.42) - /unlink 3117 (time 1788626523.19 total 20.02 last 146.87) total: 4500 /unlink in 28.82 seconds: 156.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 941 (time 1788626546.42 total 10.00 last 94.09) - open/close 1860 (time 1788626556.42 total 20.00 last 91.87) - open/close 2823 (time 1788626566.42 total 30.01 last 96.27) - open/close 3744 (time 1788626576.43 total 40.01 last 92.04) total: 4500 open/close in 48.08 seconds: 93.60 ops/second - /unlink 1737 (time 1788626598.62 total 10.01 last 173.57) - /unlink 3554 (time 1788626608.62 total 20.01 last 181.59) total: 4500 /unlink in 25.26 seconds: 178.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 927 (time 1788626627.38 total 10.00 last 92.66) - open/close 1819 (time 1788626637.38 total 20.01 last 89.19) - open/close 2726 (time 1788626647.39 total 30.01 last 90.62) - open/close 3633 (time 1788626657.40 total 40.03 last 90.59) total: 4500 open/close in 49.91 seconds: 90.17 ops/second - /unlink 1833 (time 1788626681.64 total 10.00 last 183.22) - /unlink 3319 (time 1788626691.64 total 20.01 last 148.58) total: 4500 /unlink in 27.67 seconds: 162.65 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95099 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg608-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server unloading modules via unload_modules_local on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing unload_modules_local oleg608-server: kmemleak disabled modules unloaded.