== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 14:15:40 (1788459340) Stopping clients: oleg320-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg320-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg320-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg320-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg320-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg320-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg320-server oleg320-server: oleg320-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg320-server' oleg320-server: oleg320-server.virtnet: executing load_modules_local oleg320-server: kmemleak disabled oleg320-server: Loading modules from /home/green/git/lustre-release/lustre oleg320-server: detected 4 online CPUs by sysfs oleg320-server: MODOPTS_LIBCFS= oleg320-server: Force libcfs to create 2 CPU partitions oleg320-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg320-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg320-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg320-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg320-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg320-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg320-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg320-client.virtnet: -o user_xattr,flock 192.168.203.120@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 805 (time 1788459556.44 total 10.00 last 80.46) - open/close 1676 (time 1788459566.46 total 20.02 last 87.00) - open/close 2529 (time 1788459576.46 total 30.02 last 85.29) - open/close 3376 (time 1788459586.47 total 40.04 last 84.55) - open/close 4250 (time 1788459596.48 total 50.04 last 87.35) total: 4500 open/close in 52.97 seconds: 84.95 ops/second - /unlink 1537 (time 1788459613.67 total 10.00 last 153.69) - /unlink 3081 (time 1788459623.68 total 20.00 last 154.35) total: 4500 /unlink in 29.72 seconds: 151.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 879 (time 1788459647.37 total 10.01 last 87.84) - open/close 1800 (time 1788459657.38 total 20.01 last 92.06) - open/close 2622 (time 1788459667.38 total 30.02 last 82.12) - open/close 3527 (time 1788459677.39 total 40.03 last 90.43) - open/close 4372 (time 1788459687.40 total 50.04 last 84.43) total: 4500 open/close in 51.34 seconds: 87.65 ops/second - /unlink 1622 (time 1788459702.47 total 10.00 last 162.17) - /unlink 3166 (time 1788459712.48 total 20.01 last 154.29) total: 4500 /unlink in 28.39 seconds: 158.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 799 (time 1788459735.59 total 10.00 last 79.86) - open/close 1569 (time 1788459745.60 total 20.01 last 76.99) - open/close 2311 (time 1788459755.60 total 30.01 last 74.16) - open/close 3089 (time 1788459765.60 total 40.01 last 77.78) - open/close 3863 (time 1788459775.61 total 50.02 last 77.37) total: 4500 open/close in 57.76 seconds: 77.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1555 (time 1788459798.28 total 10.00 last 155.47) - /unlink 3045 (time 1788459808.28 total 20.00 last 148.98) total: 4500 /unlink in 29.46 seconds: 152.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 848 (time 1788459832.56 total 10.00 last 84.76) - open/close 1653 (time 1788459842.57 total 20.02 last 80.41) - open/close 2408 (time 1788459852.57 total 30.02 last 75.47) - open/close 3276 (time 1788459862.59 total 40.03 last 86.67) - open/close 4141 (time 1788459872.59 total 50.04 last 86.48) total: 4500 open/close in 54.60 seconds: 82.41 ops/second - /unlink 1388 (time 1788459891.68 total 10.01 last 138.68) - /unlink 2903 (time 1788459901.68 total 20.01 last 151.46) - /unlink 4445 (time 1788459911.69 total 30.01 last 154.15) total: 4500 /unlink in 30.59 seconds: 147.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 748 (time 1788459926.58 total 10.02 last 74.68) - open/close 1542 (time 1788459936.59 total 20.02 last 79.36) - open/close 2360 (time 1788459946.59 total 30.03 last 81.75) - open/close 3159 (time 1788459956.59 total 40.03 last 79.90) - open/close 4026 (time 1788459966.62 total 50.05 last 86.49) total: 4500 open/close in 55.77 seconds: 80.68 ops/second - /unlink 1404 (time 1788459986.87 total 10.00 last 140.38) - /unlink 2801 (time 1788459996.89 total 20.02 last 139.38) - /unlink 4294 (time 1788460006.89 total 30.03 last 149.23) total: 4500 /unlink in 31.71 seconds: 141.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 802 (time 1788460023.57 total 10.01 last 80.09) - open/close 1576 (time 1788460033.58 total 20.02 last 77.34) - open/close 2286 (time 1788460043.60 total 30.04 last 70.84) - open/close 3126 (time 1788460053.62 total 40.06 last 83.85) - open/close 3917 (time 1788460063.62 total 50.07 last 79.07) total: 4500 open/close in 57.61 seconds: 78.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1631 (time 1788460085.73 total 10.00 last 163.08) - /unlink 3157 (time 1788460095.74 total 20.01 last 152.48) total: 4500 /unlink in 28.51 seconds: 157.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 831 (time 1788460119.29 total 10.02 last 82.90) - open/close 1587 (time 1788460129.30 total 20.03 last 75.53) - open/close 2397 (time 1788460139.30 total 30.03 last 81.00) - open/close 3209 (time 1788460149.33 total 40.06 last 81.01) - open/close 4032 (time 1788460159.33 total 50.06 last 82.27) total: 4500 open/close in 56.01 seconds: 80.35 ops/second - /unlink 1387 (time 1788460179.77 total 10.00 last 138.69) - /unlink 2636 (time 1788460189.77 total 20.00 last 124.89) - /unlink 4102 (time 1788460199.77 total 30.01 last 146.55) total: 4500 /unlink in 32.64 seconds: 137.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 755 (time 1788460217.25 total 10.01 last 75.41) - open/close 1623 (time 1788460227.25 total 20.02 last 86.77) - open/close 2402 (time 1788460237.27 total 30.03 last 77.80) - open/close 3206 (time 1788460247.27 total 40.03 last 80.39) - open/close 4053 (time 1788460257.27 total 50.03 last 84.68) total: 4500 open/close in 55.35 seconds: 81.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1651 (time 1788460277.08 total 10.00 last 165.03) - /unlink 3089 (time 1788460287.08 total 20.00 last 143.80) total: 4500 /unlink in 29.64 seconds: 151.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 864 (time 1788460311.45 total 10.01 last 86.28) - open/close 1618 (time 1788460321.45 total 20.02 last 75.36) - open/close 2450 (time 1788460331.46 total 30.02 last 83.16) - open/close 3275 (time 1788460341.46 total 40.03 last 82.48) - open/close 4091 (time 1788460351.47 total 50.04 last 81.52) total: 4500 open/close in 55.34 seconds: 81.32 ops/second - /unlink 1664 (time 1788460370.71 total 10.00 last 166.36) - /unlink 3273 (time 1788460380.71 total 20.01 last 160.80) total: 4500 /unlink in 28.54 seconds: 157.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 825 (time 1788460404.15 total 10.00 last 82.48) - open/close 1645 (time 1788460414.15 total 20.00 last 81.99) - open/close 2429 (time 1788460424.15 total 30.01 last 78.38) - open/close 3210 (time 1788460434.16 total 40.01 last 78.07) - open/close 3956 (time 1788460444.16 total 50.01 last 74.60) total: 4500 open/close in 57.67 seconds: 78.03 ops/second - /unlink 1681 (time 1788460466.42 total 10.01 last 167.97) - /unlink 3211 (time 1788460476.43 total 20.01 last 152.92) total: 4500 /unlink in 28.01 seconds: 160.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 811 (time 1788460499.35 total 10.01 last 81.02) - open/close 1621 (time 1788460509.35 total 20.01 last 80.99) - open/close 2397 (time 1788460519.35 total 30.01 last 77.58) - open/close 3206 (time 1788460529.36 total 40.02 last 80.86) - open/close 4054 (time 1788460539.36 total 50.02 last 84.79) total: 4500 open/close in 55.19 seconds: 81.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1512 (time 1788460558.77 total 10.00 last 151.17) - /unlink 2952 (time 1788460568.77 total 20.01 last 143.96) - /unlink 4374 (time 1788460578.77 total 30.01 last 142.19) total: 4500 /unlink in 31.09 seconds: 144.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 780 (time 1788460594.75 total 10.00 last 77.96) - open/close 1641 (time 1788460604.76 total 20.01 last 86.07) - open/close 2501 (time 1788460614.78 total 30.03 last 85.83) - open/close 3296 (time 1788460624.78 total 40.03 last 79.47) - open/close 4125 (time 1788460634.79 total 50.04 last 82.82) total: 4500 open/close in 54.95 seconds: 81.90 ops/second - /unlink 1542 (time 1788460653.59 total 10.00 last 154.17) - /unlink 2982 (time 1788460663.59 total 20.00 last 143.98) - /unlink 4434 (time 1788460673.60 total 30.01 last 145.14) total: 4500 /unlink in 30.40 seconds: 148.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 789 (time 1788460689.48 total 10.01 last 78.78) - open/close 1617 (time 1788460699.49 total 20.02 last 82.76) - open/close 2429 (time 1788460709.49 total 30.03 last 81.16) - open/close 3309 (time 1788460719.50 total 40.03 last 87.98) - open/close 4055 (time 1788460729.50 total 50.03 last 74.59) total: 4500 open/close in 55.47 seconds: 81.12 ops/second - /unlink 1596 (time 1788460749.01 total 10.00 last 159.59) - /unlink 3180 (time 1788460759.01 total 20.00 last 158.33) total: 4500 /unlink in 28.76 seconds: 156.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 803 (time 1788460782.90 total 10.00 last 80.27) - open/close 1574 (time 1788460792.91 total 20.01 last 77.04) - open/close 2345 (time 1788460802.93 total 30.03 last 76.93) - open/close 3163 (time 1788460812.94 total 40.04 last 81.75) - open/close 3908 (time 1788460822.94 total 50.04 last 74.48) total: 4500 open/close in 57.58 seconds: 78.15 ops/second - /unlink 1631 (time 1788460844.24 total 10.00 last 163.04) - /unlink 3250 (time 1788460854.25 total 20.01 last 161.87) total: 4500 /unlink in 28.00 seconds: 160.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 650 (time 1788460877.09 total 10.00 last 64.97) - open/close 1273 (time 1788460887.09 total 20.01 last 62.28) - open/close 2070 (time 1788460897.10 total 30.01 last 79.66) - open/close 2899 (time 1788460907.10 total 40.02 last 82.83) - open/close 3841 (time 1788460917.11 total 50.03 last 94.14) total: 4500 open/close in 58.20 seconds: 77.32 ops/second - /unlink 1630 (time 1788460939.22 total 10.01 last 162.90) - /unlink 3146 (time 1788460949.22 total 20.01 last 151.56) total: 4500 /unlink in 28.84 seconds: 156.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 837 (time 1788460972.52 total 10.01 last 83.63) - open/close 1613 (time 1788460982.52 total 20.01 last 77.56) - open/close 2385 (time 1788460992.53 total 30.01 last 77.19) - open/close 3093 (time 1788461002.53 total 40.02 last 70.74) - open/close 3872 (time 1788461012.54 total 50.03 last 77.82) total: 4500 open/close in 58.02 seconds: 77.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1558 (time 1788461034.38 total 10.01 last 155.68) - /unlink 3019 (time 1788461044.39 total 20.02 last 145.93) total: 4500 /unlink in 29.18 seconds: 154.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 810 (time 1788461068.47 total 10.01 last 80.96) - open/close 1596 (time 1788461078.48 total 20.01 last 78.56) - open/close 2401 (time 1788461088.48 total 30.01 last 80.47) - open/close 3216 (time 1788461098.50 total 40.04 last 81.32) - open/close 3976 (time 1788461108.51 total 50.04 last 75.99) total: 4500 open/close in 56.38 seconds: 79.81 ops/second - /unlink 1664 (time 1788461129.33 total 10.00 last 166.34) - /unlink 3111 (time 1788461139.34 total 20.02 last 144.52) total: 4500 /unlink in 28.90 seconds: 155.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 807 (time 1788461162.47 total 10.03 last 80.50) - open/close 1570 (time 1788461172.47 total 20.03 last 76.29) - open/close 2314 (time 1788461182.48 total 30.03 last 74.37) - open/close 3136 (time 1788461192.49 total 40.04 last 82.11) - open/close 3981 (time 1788461202.50 total 50.06 last 84.35) total: 4500 open/close in 55.90 seconds: 80.50 ops/second - /unlink 1473 (time 1788461222.50 total 10.00 last 147.26) - /unlink 2969 (time 1788461232.51 total 20.01 last 149.45) - /unlink 4158 (time 1788461242.51 total 30.02 last 118.87) total: 4500 /unlink in 33.19 seconds: 135.56 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95297 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg320-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg320-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg320-server stop mds service on oleg320-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg320-server stop mds service on oleg320-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg320-server unloading modules via unload_modules_local on: 'oleg320-server' oleg320-server: oleg320-server.virtnet: executing unload_modules_local oleg320-server: kmemleak disabled modules unloaded.