== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 17:52:50 (1788472370) Stopping clients: oleg302-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg302-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg302-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg302-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg302-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg302-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg302-server oleg302-server: oleg302-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing load_modules_local oleg302-server: kmemleak disabled oleg302-server: Loading modules from /home/green/git/lustre-release/lustre oleg302-server: detected 4 online CPUs by sysfs oleg302-server: MODOPTS_LIBCFS= oleg302-server: Force libcfs to create 2 CPU partitions oleg302-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg302-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg302-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg302-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg302-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg302-client.virtnet: -o user_xattr,flock 192.168.203.102@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 627 (time 1788472547.80 total 10.00 last 62.67) - open/close 1298 (time 1788472557.81 total 20.01 last 67.07) - open/close 2117 (time 1788472567.81 total 30.01 last 81.85) - open/close 3000 (time 1788472577.82 total 40.02 last 88.26) - open/close 3864 (time 1788472587.82 total 50.03 last 86.35) total: 4500 open/close in 56.88 seconds: 79.11 ops/second - /unlink 1718 (time 1788472608.25 total 10.00 last 171.77) - /unlink 3303 (time 1788472618.25 total 20.01 last 158.44) total: 4500 /unlink in 27.30 seconds: 164.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 900 (time 1788472639.57 total 10.00 last 89.99) - open/close 1738 (time 1788472649.58 total 20.00 last 83.77) - open/close 2665 (time 1788472659.58 total 30.01 last 92.68) - open/close 3602 (time 1788472669.58 total 40.01 last 93.70) total: 4500 open/close in 49.42 seconds: 91.06 ops/second - /unlink 1770 (time 1788472692.09 total 10.00 last 176.98) - /unlink 3334 (time 1788472702.09 total 20.01 last 156.33) total: 4500 /unlink in 26.85 seconds: 167.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 819 (time 1788472722.61 total 10.02 last 81.77) - open/close 1609 (time 1788472732.61 total 20.02 last 78.97) - open/close 2476 (time 1788472742.62 total 30.03 last 86.62) - open/close 3263 (time 1788472752.63 total 40.03 last 78.68) - open/close 4105 (time 1788472762.64 total 50.04 last 84.11) total: 4500 open/close in 54.57 seconds: 82.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1675 (time 1788472780.77 total 10.00 last 167.44) - /unlink 3396 (time 1788472790.78 total 20.01 last 172.01) total: 4500 /unlink in 27.33 seconds: 164.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 866 (time 1788472811.87 total 10.01 last 86.50) - open/close 1684 (time 1788472821.88 total 20.02 last 81.76) - open/close 2525 (time 1788472831.88 total 30.02 last 84.09) - open/close 3344 (time 1788472841.88 total 40.02 last 81.87) - open/close 4186 (time 1788472851.89 total 50.03 last 84.14) total: 4500 open/close in 54.18 seconds: 83.06 ops/second - /unlink 1493 (time 1788472869.86 total 10.00 last 149.29) - /unlink 3064 (time 1788472879.86 total 20.00 last 157.10) total: 4500 /unlink in 29.07 seconds: 154.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 861 (time 1788472903.52 total 10.01 last 86.04) - open/close 1722 (time 1788472913.52 total 20.01 last 86.10) - open/close 2585 (time 1788472923.53 total 30.01 last 86.27) - open/close 3438 (time 1788472933.53 total 40.01 last 85.28) - open/close 4257 (time 1788472943.54 total 50.02 last 81.83) total: 4500 open/close in 53.66 seconds: 83.86 ops/second - /unlink 1499 (time 1788472961.47 total 10.00 last 149.86) - /unlink 3033 (time 1788472971.48 total 20.01 last 153.29) total: 4500 /unlink in 29.87 seconds: 150.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 799 (time 1788472996.78 total 10.01 last 79.85) - open/close 1628 (time 1788473006.79 total 20.02 last 82.80) - open/close 2517 (time 1788473016.80 total 30.03 last 88.84) - open/close 3381 (time 1788473026.80 total 40.03 last 86.35) - open/close 4312 (time 1788473036.81 total 50.04 last 93.04) total: 4500 open/close in 52.01 seconds: 86.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1823 (time 1788473051.89 total 10.01 last 182.21) - /unlink 3474 (time 1788473061.89 total 20.01 last 165.08) total: 4500 /unlink in 26.54 seconds: 169.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 809 (time 1788473082.54 total 10.00 last 80.88) - open/close 1692 (time 1788473092.55 total 20.01 last 88.23) - open/close 2515 (time 1788473102.56 total 30.02 last 82.24) - open/close 3344 (time 1788473112.58 total 40.04 last 82.76) - open/close 4149 (time 1788473122.58 total 50.04 last 80.46) total: 4500 open/close in 53.76 seconds: 83.71 ops/second - /unlink 1567 (time 1788473140.74 total 10.00 last 156.69) - /unlink 3069 (time 1788473150.74 total 20.00 last 150.14) total: 4500 /unlink in 29.51 seconds: 152.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 874 (time 1788473175.80 total 10.02 last 87.26) - open/close 1675 (time 1788473185.81 total 20.03 last 80.02) - open/close 2459 (time 1788473195.82 total 30.04 last 78.32) - open/close 3239 (time 1788473205.83 total 40.04 last 77.94) - open/close 4122 (time 1788473215.83 total 50.05 last 88.28) total: 4500 open/close in 54.52 seconds: 82.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1598 (time 1788473233.90 total 10.00 last 159.75) - /unlink 3118 (time 1788473243.90 total 20.01 last 151.93) total: 4500 /unlink in 29.02 seconds: 155.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 764 (time 1788473267.55 total 10.00 last 76.38) - open/close 1427 (time 1788473277.56 total 20.01 last 66.26) - open/close 2164 (time 1788473287.56 total 30.01 last 73.68) - open/close 3065 (time 1788473297.56 total 40.02 last 90.05) - open/close 3941 (time 1788473307.57 total 50.02 last 87.54) total: 4500 open/close in 56.49 seconds: 79.66 ops/second - /unlink 1555 (time 1788473327.66 total 10.00 last 155.49) - /unlink 3092 (time 1788473337.66 total 20.00 last 153.66) - /unlink 4426 (time 1788473347.68 total 30.01 last 133.24) total: 4500 /unlink in 30.71 seconds: 146.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 801 (time 1788473362.83 total 10.01 last 80.03) - open/close 1577 (time 1788473372.83 total 20.01 last 77.60) - open/close 2394 (time 1788473382.84 total 30.02 last 81.60) - open/close 3118 (time 1788473392.84 total 40.02 last 72.40) - open/close 3921 (time 1788473402.86 total 50.04 last 80.17) total: 4500 open/close in 56.86 seconds: 79.15 ops/second - /unlink 1587 (time 1788473424.21 total 10.01 last 158.54) - /unlink 2924 (time 1788473434.21 total 20.01 last 133.66) - /unlink 4469 (time 1788473444.21 total 30.01 last 154.50) total: 4500 /unlink in 30.34 seconds: 148.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 803 (time 1788473458.53 total 10.00 last 80.27) - open/close 1547 (time 1788473468.53 total 20.01 last 74.36) - open/close 2355 (time 1788473478.54 total 30.01 last 80.78) - open/close 3146 (time 1788473488.55 total 40.02 last 79.02) - open/close 3960 (time 1788473498.56 total 50.03 last 81.32) total: 4500 open/close in 57.01 seconds: 78.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1601 (time 1788473520.34 total 10.00 last 160.09) - /unlink 3151 (time 1788473530.34 total 20.00 last 154.98) total: 4500 /unlink in 28.78 seconds: 156.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 799 (time 1788473553.72 total 10.00 last 79.87) - open/close 1670 (time 1788473563.72 total 20.01 last 87.09) - open/close 2502 (time 1788473573.73 total 30.01 last 83.16) - open/close 3353 (time 1788473583.73 total 40.02 last 85.04) - open/close 4173 (time 1788473593.75 total 50.03 last 81.87) total: 4500 open/close in 54.11 seconds: 83.17 ops/second - /unlink 1515 (time 1788473611.94 total 10.00 last 151.49) - /unlink 2977 (time 1788473621.94 total 20.00 last 146.19) total: 4500 /unlink in 29.86 seconds: 150.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 837 (time 1788473646.14 total 10.01 last 83.63) - open/close 1693 (time 1788473656.14 total 20.01 last 85.56) - open/close 2549 (time 1788473666.15 total 30.02 last 85.58) - open/close 3424 (time 1788473676.16 total 40.03 last 87.42) - open/close 4175 (time 1788473686.16 total 50.03 last 75.07) total: 4500 open/close in 53.80 seconds: 83.64 ops/second - /unlink 1469 (time 1788473703.49 total 10.00 last 146.87) - /unlink 2856 (time 1788473713.49 total 20.00 last 138.69) - /unlink 4135 (time 1788473723.49 total 30.01 last 127.86) total: 4500 /unlink in 32.66 seconds: 137.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 816 (time 1788473740.60 total 10.00 last 81.57) - open/close 1557 (time 1788473750.61 total 20.01 last 74.06) - open/close 2330 (time 1788473760.62 total 30.02 last 77.23) - open/close 3029 (time 1788473770.62 total 40.02 last 69.86) - open/close 3715 (time 1788473780.62 total 50.02 last 68.60) - open/close 4406 (time 1788473790.63 total 60.03 last 69.06) total: 4500 open/close in 61.17 seconds: 73.56 ops/second - /unlink 1455 (time 1788473805.69 total 10.00 last 145.44) - /unlink 2910 (time 1788473815.69 total 20.00 last 145.49) total: 4500 /unlink in 29.96 seconds: 150.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 785 (time 1788473840.11 total 10.00 last 78.48) - open/close 1523 (time 1788473850.11 total 20.01 last 73.74) - open/close 2285 (time 1788473860.11 total 30.01 last 76.20) - open/close 3024 (time 1788473870.13 total 40.03 last 73.78) - open/close 3873 (time 1788473880.14 total 50.03 last 84.86) total: 4500 open/close in 57.31 seconds: 78.52 ops/second - /unlink 1515 (time 1788473901.14 total 10.00 last 151.43) - /unlink 2924 (time 1788473911.15 total 20.01 last 140.77) - /unlink 4335 (time 1788473921.16 total 30.02 last 140.99) total: 4500 /unlink in 31.33 seconds: 143.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 787 (time 1788473937.67 total 10.01 last 78.62) - open/close 1549 (time 1788473947.69 total 20.03 last 76.08) - open/close 2303 (time 1788473957.70 total 30.04 last 75.33) - open/close 3079 (time 1788473967.71 total 40.05 last 77.51) - open/close 3947 (time 1788473977.72 total 50.05 last 86.74) total: 4500 open/close in 56.02 seconds: 80.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1549 (time 1788473997.49 total 10.00 last 154.85) - /unlink 3032 (time 1788474007.50 total 20.01 last 148.25) total: 4500 /unlink in 28.51 seconds: 157.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 891 (time 1788474029.96 total 10.01 last 89.03) - open/close 1730 (time 1788474039.97 total 20.02 last 83.83) - open/close 2510 (time 1788474049.99 total 30.04 last 77.84) - open/close 3237 (time 1788474060.00 total 40.05 last 72.63) - open/close 4050 (time 1788474070.01 total 50.05 last 81.24) total: 4500 open/close in 56.00 seconds: 80.36 ops/second - /unlink 1463 (time 1788474090.22 total 10.00 last 146.30) - /unlink 2941 (time 1788474100.23 total 20.01 last 147.66) - /unlink 4082 (time 1788474110.24 total 30.02 last 113.99) total: 4500 /unlink in 33.83 seconds: 133.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 840 (time 1788474129.12 total 10.01 last 83.94) - open/close 1686 (time 1788474139.12 total 20.01 last 84.53) - open/close 2363 (time 1788474149.14 total 30.03 last 67.63) - open/close 3198 (time 1788474159.14 total 40.03 last 83.42) - open/close 4026 (time 1788474169.15 total 50.04 last 82.72) total: 4500 open/close in 56.11 seconds: 80.20 ops/second - /unlink 1495 (time 1788474190.00 total 10.00 last 149.49) - /unlink 2957 (time 1788474200.00 total 20.00 last 146.18) - /unlink 4380 (time 1788474210.00 total 30.01 last 142.25) total: 4500 /unlink in 30.95 seconds: 145.39 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95199 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg302-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg302-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg302-server stop mds service on oleg302-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg302-server stop mds service on oleg302-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg302-server unloading modules via unload_modules_local on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing unload_modules_local oleg302-server: kmemleak disabled modules unloaded.