== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 13:57:22 (1788803842) Stopping clients: oleg656-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg656-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg656-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg656-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg656-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg656-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg656-server oleg656-server: oleg656-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg656-server' oleg656-server: oleg656-server.virtnet: executing load_modules_local oleg656-server: kmemleak disabled oleg656-server: Loading modules from /home/green/git/lustre-release/lustre oleg656-server: detected 4 online CPUs by sysfs oleg656-server: MODOPTS_LIBCFS= oleg656-server: Force libcfs to create 2 CPU partitions oleg656-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg656-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg656-server: oleg656-server.virtnet: executing set_default_debug -1 all pdsh@oleg656-client: oleg656-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg656-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg656-server: oleg656-server.virtnet: executing set_default_debug -1 all pdsh@oleg656-client: oleg656-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg656-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg656-server: oleg656-server.virtnet: executing set_default_debug -1 all pdsh@oleg656-client: oleg656-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg656-client.virtnet: -o user_xattr,flock 192.168.206.156@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 598 (time 1788804039.06 total 10.02 last 59.70) - open/close 1239 (time 1788804049.07 total 20.02 last 64.06) - open/close 1833 (time 1788804059.08 total 30.03 last 59.34) - open/close 2474 (time 1788804069.09 total 40.04 last 64.04) - open/close 3129 (time 1788804079.10 total 50.05 last 65.43) - open/close 3824 (time 1788804089.11 total 60.06 last 69.45) - open/close 4474 (time 1788804099.12 total 70.07 last 64.94) total: 4500 open/close in 70.38 seconds: 63.94 ops/second - /unlink 1233 (time 1788804114.32 total 10.01 last 123.15) - /unlink 2446 (time 1788804124.32 total 20.02 last 121.27) - /unlink 3478 (time 1788804134.33 total 30.02 last 103.14) total: 4500 /unlink in 39.76 seconds: 113.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 720 (time 1788804159.96 total 10.01 last 71.96) - open/close 1383 (time 1788804169.98 total 20.02 last 66.20) - open/close 2097 (time 1788804179.99 total 30.04 last 71.29) - open/close 2798 (time 1788804190.01 total 40.05 last 69.99) - open/close 3418 (time 1788804200.03 total 50.07 last 61.86) - open/close 4125 (time 1788804210.03 total 60.08 last 70.70) total: 4500 open/close in 65.69 seconds: 68.50 ops/second - /unlink 1186 (time 1788804230.41 total 10.00 last 118.59) - /unlink 2479 (time 1788804240.42 total 20.01 last 129.23) - /unlink 3762 (time 1788804250.42 total 30.01 last 128.26) total: 4500 /unlink in 35.63 seconds: 126.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 685 (time 1788804270.70 total 10.01 last 68.45) - open/close 1350 (time 1788804280.71 total 20.01 last 66.45) - open/close 1974 (time 1788804290.72 total 30.03 last 62.33) - open/close 2719 (time 1788804300.72 total 40.03 last 74.50) - open/close 3454 (time 1788804310.73 total 50.03 last 73.43) - open/close 4099 (time 1788804320.74 total 60.05 last 64.42) total: 4500 open/close in 66.18 seconds: 67.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1278 (time 1788804341.35 total 10.01 last 127.73) - /unlink 2467 (time 1788804351.35 total 20.01 last 118.86) - /unlink 3732 (time 1788804361.36 total 30.02 last 126.36) total: 4500 /unlink in 36.56 seconds: 123.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 599 (time 1788804382.95 total 10.01 last 59.83) - open/close 1234 (time 1788804392.96 total 20.02 last 63.45) - open/close 1905 (time 1788804402.97 total 30.03 last 67.04) - open/close 2594 (time 1788804412.98 total 40.04 last 68.80) - open/close 3250 (time 1788804422.99 total 50.05 last 65.58) - open/close 3939 (time 1788804432.99 total 60.05 last 68.87) total: 4500 open/close in 68.57 seconds: 65.63 ops/second - /unlink 1319 (time 1788804455.89 total 10.00 last 131.88) - /unlink 2664 (time 1788804465.89 total 20.00 last 134.50) - /unlink 3913 (time 1788804475.89 total 30.00 last 124.88) total: 4500 /unlink in 34.87 seconds: 129.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 617 (time 1788804495.15 total 10.03 last 61.51) - open/close 1318 (time 1788804505.15 total 20.03 last 70.08) - open/close 1988 (time 1788804515.16 total 30.05 last 66.92) - open/close 2661 (time 1788804525.20 total 40.08 last 67.08) - open/close 3329 (time 1788804535.21 total 50.10 last 66.69) - open/close 4013 (time 1788804545.22 total 60.10 last 68.37) total: 4500 open/close in 66.67 seconds: 67.50 ops/second - /unlink 1246 (time 1788804566.87 total 10.00 last 124.58) - /unlink 2476 (time 1788804576.90 total 20.02 last 122.73) - /unlink 3713 (time 1788804586.90 total 30.03 last 123.65) total: 4500 /unlink in 37.05 seconds: 121.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 714 (time 1788804609.04 total 10.01 last 71.33) - open/close 1366 (time 1788804619.05 total 20.01 last 65.17) - open/close 2014 (time 1788804629.05 total 30.02 last 64.78) - open/close 2681 (time 1788804639.06 total 40.03 last 66.64) - open/close 3336 (time 1788804649.06 total 50.03 last 65.47) - open/close 4087 (time 1788804659.06 total 60.03 last 75.09) total: 4500 open/close in 66.42 seconds: 67.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 869 (time 1788804679.94 total 10.01 last 86.81) - /unlink 1778 (time 1788804689.95 total 20.02 last 90.78) - /unlink 2992 (time 1788804699.96 total 30.02 last 121.39) - /unlink 4444 (time 1788804709.96 total 40.03 last 145.19) total: 4500 /unlink in 40.50 seconds: 111.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 763 (time 1788804724.78 total 10.01 last 76.26) - open/close 1505 (time 1788804734.79 total 20.01 last 74.16) - open/close 2242 (time 1788804744.80 total 30.02 last 73.61) - open/close 2997 (time 1788804754.80 total 40.03 last 75.45) - open/close 3703 (time 1788804764.82 total 50.05 last 70.49) - open/close 4458 (time 1788804774.83 total 60.05 last 75.46) total: 4500 open/close in 60.57 seconds: 74.30 ops/second - /unlink 1487 (time 1788804789.49 total 10.00 last 148.69) - /unlink 2666 (time 1788804799.49 total 20.00 last 117.89) - /unlink 4002 (time 1788804809.50 total 30.00 last 133.56) total: 4500 /unlink in 34.07 seconds: 132.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 649 (time 1788804828.21 total 10.00 last 64.88) - open/close 1358 (time 1788804838.22 total 20.01 last 70.84) - open/close 2058 (time 1788804848.23 total 30.02 last 69.91) - open/close 2835 (time 1788804858.25 total 40.04 last 77.60) - open/close 3592 (time 1788804868.26 total 50.05 last 75.57) - open/close 4285 (time 1788804878.27 total 60.06 last 69.25) total: 4500 open/close in 63.00 seconds: 71.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1342 (time 1788804896.05 total 10.00 last 134.18) - /unlink 2666 (time 1788804906.05 total 20.00 last 132.39) - /unlink 3930 (time 1788804916.06 total 30.00 last 126.36) total: 4500 /unlink in 34.78 seconds: 129.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 660 (time 1788804935.22 total 10.01 last 65.93) - open/close 1362 (time 1788804945.23 total 20.02 last 70.16) - open/close 2114 (time 1788804955.23 total 30.02 last 75.19) - open/close 2953 (time 1788804965.23 total 40.02 last 83.90) - open/close 3791 (time 1788804975.23 total 50.02 last 83.75) total: 4500 open/close in 59.24 seconds: 75.96 ops/second - /unlink 1461 (time 1788804997.50 total 10.00 last 146.03) - /unlink 2918 (time 1788805007.50 total 20.01 last 145.67) - /unlink 4353 (time 1788805017.51 total 30.01 last 143.40) total: 4500 /unlink in 31.08 seconds: 144.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 770 (time 1788805031.91 total 10.00 last 76.97) - open/close 1564 (time 1788805041.92 total 20.01 last 79.33) - open/close 2284 (time 1788805051.92 total 30.02 last 71.97) - open/close 3048 (time 1788805061.93 total 40.03 last 76.33) - open/close 3874 (time 1788805071.93 total 50.03 last 82.57) total: 4500 open/close in 58.35 seconds: 77.12 ops/second - /unlink 1563 (time 1788805093.61 total 10.00 last 156.26) - /unlink 3152 (time 1788805103.61 total 20.01 last 158.83) total: 4500 /unlink in 29.41 seconds: 152.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 744 (time 1788805126.98 total 10.00 last 74.39) - open/close 1491 (time 1788805136.98 total 20.00 last 74.68) - open/close 2322 (time 1788805146.99 total 30.02 last 82.97) - open/close 3095 (time 1788805157.00 total 40.02 last 77.27) - open/close 3826 (time 1788805167.00 total 50.03 last 73.08) total: 4500 open/close in 58.81 seconds: 76.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1526 (time 1788805189.11 total 10.00 last 152.59) - /unlink 3029 (time 1788805199.12 total 20.01 last 150.09) total: 4500 /unlink in 29.42 seconds: 152.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 829 (time 1788805222.94 total 10.00 last 82.87) - open/close 1649 (time 1788805232.94 total 20.01 last 81.96) - open/close 2452 (time 1788805242.95 total 30.01 last 80.26) - open/close 3236 (time 1788805252.95 total 40.02 last 78.38) - open/close 4008 (time 1788805262.95 total 50.02 last 77.18) total: 4500 open/close in 56.27 seconds: 79.97 ops/second - /unlink 1557 (time 1788805282.33 total 10.00 last 155.65) - /unlink 3077 (time 1788805292.33 total 20.01 last 151.95) total: 4500 /unlink in 29.36 seconds: 153.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 784 (time 1788805315.85 total 10.01 last 78.35) - open/close 1566 (time 1788805325.86 total 20.01 last 78.16) - open/close 2437 (time 1788805335.86 total 30.01 last 87.08) - open/close 3307 (time 1788805345.86 total 40.01 last 86.99) - open/close 4165 (time 1788805355.87 total 50.02 last 85.77) total: 4500 open/close in 54.00 seconds: 83.33 ops/second - /unlink 1609 (time 1788805372.79 total 10.00 last 160.84) - /unlink 3203 (time 1788805382.80 total 20.01 last 159.33) total: 4500 /unlink in 27.93 seconds: 161.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 862 (time 1788805404.47 total 10.00 last 86.20) - open/close 1710 (time 1788805414.47 total 20.01 last 84.75) - open/close 2616 (time 1788805424.47 total 30.01 last 90.59) - open/close 3612 (time 1788805434.48 total 40.02 last 99.52) total: 4500 open/close in 49.69 seconds: 90.56 ops/second - /unlink 1945 (time 1788805456.93 total 10.00 last 194.47) - /unlink 3710 (time 1788805466.93 total 20.01 last 176.39) total: 4500 /unlink in 24.56 seconds: 183.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 829 (time 1788805484.60 total 10.00 last 82.89) - open/close 1666 (time 1788805494.61 total 20.01 last 83.60) - open/close 2491 (time 1788805504.62 total 30.02 last 82.43) - open/close 3382 (time 1788805514.63 total 40.03 last 89.01) - open/close 4294 (time 1788805524.63 total 50.03 last 91.18) total: 4500 open/close in 52.30 seconds: 86.04 ops/second - /unlink 1646 (time 1788805539.74 total 10.00 last 164.59) - /unlink 3265 (time 1788805549.74 total 20.00 last 161.89) total: 4500 /unlink in 28.14 seconds: 159.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 882 (time 1788805571.73 total 10.01 last 88.14) - open/close 1858 (time 1788805581.73 total 20.01 last 97.59) - open/close 2768 (time 1788805591.74 total 30.02 last 90.90) - open/close 3621 (time 1788805601.75 total 40.03 last 85.20) total: 4500 open/close in 49.93 seconds: 90.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1739 (time 1788805624.38 total 10.00 last 173.88) - /unlink 3341 (time 1788805634.38 total 20.00 last 160.14) total: 4500 /unlink in 27.50 seconds: 163.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 959 (time 1788805654.80 total 10.00 last 95.86) - open/close 1800 (time 1788805664.80 total 20.01 last 84.09) - open/close 2632 (time 1788805674.80 total 30.01 last 83.19) - open/close 3494 (time 1788805684.80 total 40.01 last 86.17) - open/close 4465 (time 1788805694.80 total 50.01 last 97.09) total: 4500 open/close in 50.38 seconds: 89.33 ops/second - /unlink 1748 (time 1788805707.71 total 10.01 last 174.71) - /unlink 3424 (time 1788805717.72 total 20.01 last 167.54) total: 4500 /unlink in 26.63 seconds: 168.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 909 (time 1788805737.63 total 10.00 last 90.88) - open/close 1753 (time 1788805747.64 total 20.01 last 84.36) - open/close 2575 (time 1788805757.64 total 30.01 last 82.18) - open/close 3462 (time 1788805767.64 total 40.01 last 88.68) - open/close 4464 (time 1788805777.65 total 50.02 last 100.15) total: 4500 open/close in 50.34 seconds: 89.40 ops/second - /unlink 1662 (time 1788805790.66 total 10.00 last 166.16) - /unlink 3492 (time 1788805800.67 total 20.01 last 182.83) total: 4500 /unlink in 26.03 seconds: 172.88 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 94992 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg656-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg656-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg656-server stop mds service on oleg656-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg656-server stop mds service on oleg656-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg656-server unloading modules via unload_modules_local on: 'oleg656-server' oleg656-server: oleg656-server.virtnet: executing unload_modules_local oleg656-server: kmemleak disabled modules unloaded.