== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:12:43 (1789269163) Stopping clients: oleg249-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg249-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg249-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg249-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg249-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg249-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg249-server oleg249-server: oleg249-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing load_modules_local oleg249-server: kmemleak disabled oleg249-server: Loading modules from /home/green/git/lustre-release/lustre oleg249-server: detected 4 online CPUs by sysfs oleg249-server: MODOPTS_LIBCFS= oleg249-server: Force libcfs to create 2 CPU partitions oleg249-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg249-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg249-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg249-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg249-client.virtnet: -o user_xattr,flock 192.168.202.149@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 908 (time 1789269363.94 total 10.00 last 90.76) - open/close 1760 (time 1789269373.96 total 20.02 last 85.03) - open/close 2685 (time 1789269383.97 total 30.03 last 92.42) - open/close 3601 (time 1789269393.97 total 40.04 last 91.56) total: 4500 open/close in 49.04 seconds: 91.77 ops/second - /unlink 2071 (time 1789269416.65 total 10.00 last 207.00) - /unlink 3975 (time 1789269426.65 total 20.01 last 190.36) total: 4500 /unlink in 23.04 seconds: 195.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 899 (time 1789269444.06 total 10.00 last 89.87) - open/close 1830 (time 1789269454.06 total 20.01 last 93.09) - open/close 2579 (time 1789269464.07 total 30.01 last 74.87) - open/close 3435 (time 1789269474.07 total 40.01 last 85.60) - open/close 4399 (time 1789269484.08 total 50.02 last 96.33) total: 4500 open/close in 51.03 seconds: 88.19 ops/second - /unlink 1708 (time 1789269498.81 total 10.00 last 170.78) - /unlink 3210 (time 1789269508.81 total 20.00 last 150.20) total: 4500 /unlink in 26.96 seconds: 166.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 972 (time 1789269529.06 total 10.00 last 97.20) - open/close 1931 (time 1789269539.07 total 20.01 last 95.84) - open/close 2943 (time 1789269549.07 total 30.01 last 101.19) - open/close 3934 (time 1789269559.08 total 40.01 last 99.03) total: 4500 open/close in 45.76 seconds: 98.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1742 (time 1789269578.38 total 10.00 last 174.20) - /unlink 3492 (time 1789269588.39 total 20.01 last 174.91) total: 4500 /unlink in 25.41 seconds: 177.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 994 (time 1789269608.55 total 10.01 last 99.34) - open/close 1975 (time 1789269618.55 total 20.01 last 98.04) - open/close 2933 (time 1789269628.56 total 30.02 last 95.74) - open/close 3660 (time 1789269638.56 total 40.02 last 72.67) total: 4500 open/close in 49.34 seconds: 91.21 ops/second - /unlink 1577 (time 1789269661.72 total 10.01 last 157.54) - /unlink 2965 (time 1789269671.72 total 20.01 last 138.78) - /unlink 4214 (time 1789269681.72 total 30.01 last 124.90) total: 4500 /unlink in 32.42 seconds: 138.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 619 (time 1789269700.49 total 10.01 last 61.86) - open/close 1498 (time 1789269710.49 total 20.01 last 87.89) - open/close 2401 (time 1789269720.50 total 30.01 last 90.23) - open/close 3248 (time 1789269730.52 total 40.04 last 84.52) - open/close 4149 (time 1789269740.52 total 50.04 last 90.05) total: 4500 open/close in 54.48 seconds: 82.60 ops/second - /unlink 1618 (time 1789269759.52 total 10.00 last 161.78) - /unlink 2829 (time 1789269769.52 total 20.01 last 121.05) - /unlink 4218 (time 1789269779.52 total 30.01 last 138.90) total: 4500 /unlink in 31.78 seconds: 141.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 922 (time 1789269795.42 total 10.01 last 92.14) - open/close 1843 (time 1789269805.42 total 20.01 last 92.08) - open/close 2759 (time 1789269815.44 total 30.03 last 91.41) - open/close 3472 (time 1789269825.45 total 40.03 last 71.27) - open/close 4292 (time 1789269835.45 total 50.03 last 81.98) total: 4500 open/close in 52.33 seconds: 85.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1417 (time 1789269852.84 total 10.00 last 141.66) - /unlink 3166 (time 1789269862.84 total 20.00 last 174.89) total: 4500 /unlink in 27.89 seconds: 161.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 889 (time 1789269885.92 total 10.01 last 88.83) - open/close 1847 (time 1789269895.92 total 20.01 last 95.75) - open/close 2624 (time 1789269905.93 total 30.02 last 77.67) - open/close 3402 (time 1789269915.93 total 40.02 last 77.78) - open/close 4191 (time 1789269925.95 total 50.04 last 78.77) total: 4500 open/close in 55.09 seconds: 81.68 ops/second - /unlink 1656 (time 1789269947.19 total 10.01 last 165.41) - /unlink 3513 (time 1789269957.20 total 20.01 last 185.64) total: 4500 /unlink in 25.60 seconds: 175.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 743 (time 1789269977.38 total 10.00 last 74.30) - open/close 1700 (time 1789269987.38 total 20.00 last 95.68) - open/close 2537 (time 1789269997.38 total 30.01 last 83.66) - open/close 3307 (time 1789270007.39 total 40.02 last 76.94) - open/close 4144 (time 1789270017.40 total 50.02 last 83.67) total: 4500 open/close in 53.95 seconds: 83.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1606 (time 1789270034.90 total 10.00 last 160.60) - /unlink 3261 (time 1789270044.90 total 20.00 last 165.48) total: 4500 /unlink in 27.34 seconds: 164.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 960 (time 1789270065.95 total 10.00 last 95.96) - open/close 1956 (time 1789270075.97 total 20.02 last 99.47) - open/close 2817 (time 1789270085.98 total 30.03 last 86.00) - open/close 3617 (time 1789270095.99 total 40.04 last 79.91) - open/close 4499 (time 1789270105.99 total 50.04 last 88.17) total: 4500 open/close in 50.04 seconds: 89.92 ops/second - /unlink 1631 (time 1789270120.75 total 10.00 last 163.09) - /unlink 3192 (time 1789270130.75 total 20.01 last 156.00) total: 4500 /unlink in 28.44 seconds: 158.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 932 (time 1789270153.44 total 10.01 last 93.15) - open/close 1873 (time 1789270163.44 total 20.01 last 94.06) - open/close 2773 (time 1789270173.45 total 30.01 last 89.96) - open/close 3658 (time 1789270183.45 total 40.02 last 88.45) - open/close 4380 (time 1789270193.49 total 50.06 last 71.91) total: 4500 open/close in 52.16 seconds: 86.28 ops/second - /unlink 1751 (time 1789270210.38 total 10.01 last 175.01) - /unlink 3432 (time 1789270220.39 total 20.01 last 168.05) total: 4500 /unlink in 26.39 seconds: 170.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1026 (time 1789270240.63 total 10.01 last 102.55) - open/close 1908 (time 1789270250.64 total 20.01 last 88.15) - open/close 2912 (time 1789270260.64 total 30.01 last 100.37) - open/close 3895 (time 1789270270.64 total 40.02 last 98.27) total: 4500 open/close in 46.26 seconds: 97.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1874 (time 1789270290.01 total 10.00 last 187.39) - /unlink 3664 (time 1789270300.03 total 20.01 last 178.77) total: 4500 /unlink in 26.44 seconds: 170.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1028 (time 1789270320.40 total 10.01 last 102.74) - open/close 2048 (time 1789270330.41 total 20.02 last 101.90) - open/close 3032 (time 1789270340.42 total 30.02 last 98.37) - open/close 3853 (time 1789270350.42 total 40.02 last 82.08) total: 4500 open/close in 47.37 seconds: 95.00 ops/second - /unlink 2027 (time 1789270371.33 total 10.00 last 202.68) - /unlink 3917 (time 1789270381.33 total 20.00 last 188.97) total: 4500 /unlink in 23.25 seconds: 193.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 964 (time 1789270398.30 total 10.01 last 96.28) - open/close 1963 (time 1789270408.30 total 20.02 last 99.87) - open/close 2792 (time 1789270418.32 total 30.03 last 82.77) - open/close 3711 (time 1789270428.33 total 40.04 last 91.83) total: 4500 open/close in 48.65 seconds: 92.50 ops/second - /unlink 1639 (time 1789270450.90 total 10.01 last 163.76) - /unlink 3400 (time 1789270460.90 total 20.01 last 176.05) total: 4500 /unlink in 26.88 seconds: 167.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 913 (time 1789270482.18 total 10.01 last 91.24) - open/close 1746 (time 1789270492.18 total 20.01 last 83.28) - open/close 2569 (time 1789270502.20 total 30.03 last 82.14) - open/close 3621 (time 1789270512.21 total 40.04 last 105.11) total: 4500 open/close in 49.17 seconds: 91.51 ops/second - /unlink 1730 (time 1789270534.61 total 10.00 last 172.99) - /unlink 3525 (time 1789270544.61 total 20.00 last 179.45) total: 4500 /unlink in 25.64 seconds: 175.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 824 (time 1789270564.70 total 10.00 last 82.36) - open/close 1752 (time 1789270574.71 total 20.01 last 92.73) - open/close 2720 (time 1789270584.71 total 30.01 last 96.79) - open/close 3730 (time 1789270594.72 total 40.02 last 100.96) total: 4500 open/close in 47.56 seconds: 94.61 ops/second - /unlink 1858 (time 1789270616.26 total 10.00 last 185.79) - /unlink 3727 (time 1789270626.27 total 20.01 last 186.79) total: 4500 /unlink in 24.61 seconds: 182.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 743 (time 1789270644.50 total 10.01 last 74.23) - open/close 1586 (time 1789270654.51 total 20.02 last 84.24) - open/close 2551 (time 1789270664.52 total 30.03 last 96.37) - open/close 3388 (time 1789270674.53 total 40.04 last 83.60) - open/close 4405 (time 1789270684.53 total 50.04 last 101.69) total: 4500 open/close in 50.93 seconds: 88.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1956 (time 1789270698.89 total 10.00 last 195.58) - /unlink 3577 (time 1789270708.90 total 20.01 last 161.98) total: 4500 /unlink in 25.09 seconds: 179.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 845 (time 1789270728.42 total 10.01 last 84.46) - open/close 1569 (time 1789270738.42 total 20.01 last 72.39) - open/close 2540 (time 1789270748.42 total 30.01 last 97.07) - open/close 3434 (time 1789270758.44 total 40.03 last 89.21) - open/close 4411 (time 1789270768.45 total 50.04 last 97.62) total: 4500 open/close in 50.96 seconds: 88.30 ops/second - /unlink 1544 (time 1789270783.44 total 10.00 last 154.37) - /unlink 2713 (time 1789270793.44 total 20.00 last 116.87) - /unlink 4285 (time 1789270803.45 total 30.01 last 157.11) total: 4500 /unlink in 31.64 seconds: 142.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 766 (time 1789270820.44 total 10.02 last 76.43) - open/close 1710 (time 1789270830.46 total 20.04 last 94.25) - open/close 2632 (time 1789270840.46 total 30.04 last 92.15) - open/close 3660 (time 1789270850.47 total 40.05 last 102.72) total: 4500 open/close in 49.24 seconds: 91.38 ops/second - /unlink 1567 (time 1789270873.91 total 10.00 last 156.63) - /unlink 3106 (time 1789270883.91 total 20.00 last 153.90) total: 4500 /unlink in 29.30 seconds: 153.59 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95208 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg249-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg249-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg249-server unloading modules via unload_modules_local on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing unload_modules_local oleg249-server: kmemleak disabled modules unloaded.