== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:19:07 (1788740347) Stopping clients: oleg342-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg342-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg342-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg342-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg342-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg342-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg342-server oleg342-server: oleg342-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg342-server' oleg342-server: oleg342-server.virtnet: executing load_modules_local oleg342-server: kmemleak disabled oleg342-server: Loading modules from /home/green/git/lustre-release/lustre oleg342-server: detected 4 online CPUs by sysfs oleg342-server: MODOPTS_LIBCFS= oleg342-server: Force libcfs to create 2 CPU partitions oleg342-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg342-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg342-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg342-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg342-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg342-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg342-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg342-client.virtnet: -o user_xattr,flock 192.168.203.142@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 919 (time 1788740523.01 total 10.01 last 91.84) - open/close 1760 (time 1788740533.02 total 20.01 last 84.07) - open/close 2633 (time 1788740543.02 total 30.01 last 87.28) - open/close 3564 (time 1788740553.02 total 40.02 last 93.07) - open/close 4471 (time 1788740563.04 total 50.03 last 90.56) total: 4500 open/close in 50.37 seconds: 89.33 ops/second - /unlink 1677 (time 1788740576.84 total 10.01 last 167.62) - /unlink 3345 (time 1788740586.84 total 20.01 last 166.65) total: 4500 /unlink in 25.81 seconds: 174.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 945 (time 1788740606.72 total 10.00 last 94.49) - open/close 1860 (time 1788740616.72 total 20.00 last 91.47) - open/close 2805 (time 1788740626.73 total 30.01 last 94.48) - open/close 3745 (time 1788740636.73 total 40.01 last 93.93) total: 4500 open/close in 47.32 seconds: 95.10 ops/second - /unlink 1808 (time 1788740657.12 total 10.00 last 180.79) - /unlink 3750 (time 1788740667.12 total 20.00 last 194.20) total: 4500 /unlink in 24.10 seconds: 186.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 984 (time 1788740684.20 total 10.01 last 98.32) - open/close 1806 (time 1788740694.20 total 20.01 last 82.19) - open/close 2684 (time 1788740704.22 total 30.03 last 87.66) - open/close 3622 (time 1788740714.22 total 40.03 last 93.80) total: 4500 open/close in 49.12 seconds: 91.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1893 (time 1788740737.12 total 10.00 last 189.27) - /unlink 3882 (time 1788740747.13 total 20.01 last 198.77) total: 4500 /unlink in 23.47 seconds: 191.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 827 (time 1788740764.47 total 10.00 last 82.67) - open/close 1706 (time 1788740774.47 total 20.01 last 87.89) - open/close 2627 (time 1788740784.48 total 30.01 last 92.05) - open/close 3604 (time 1788740794.49 total 40.02 last 97.62) total: 4500 open/close in 49.67 seconds: 90.59 ops/second - /unlink 1604 (time 1788740818.00 total 10.01 last 160.28) - /unlink 3237 (time 1788740828.00 total 20.01 last 163.28) total: 4500 /unlink in 28.20 seconds: 159.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 876 (time 1788740850.91 total 10.01 last 87.51) - open/close 1716 (time 1788740860.91 total 20.02 last 83.95) - open/close 2629 (time 1788740870.91 total 30.02 last 91.29) - open/close 3510 (time 1788740880.91 total 40.02 last 88.08) - open/close 4397 (time 1788740890.92 total 50.03 last 88.65) total: 4500 open/close in 51.01 seconds: 88.22 ops/second - /unlink 1351 (time 1788740905.32 total 10.00 last 135.08) - /unlink 2339 (time 1788740915.32 total 20.00 last 98.78) - /unlink 3462 (time 1788740925.32 total 30.00 last 112.28) total: 4500 /unlink in 38.59 seconds: 116.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 709 (time 1788740949.49 total 10.00 last 70.90) - open/close 1498 (time 1788740959.51 total 20.02 last 78.71) - open/close 2139 (time 1788740969.52 total 30.03 last 64.04) - open/close 2699 (time 1788740979.53 total 40.05 last 55.93) - open/close 3270 (time 1788740989.54 total 50.05 last 57.07) - open/close 3914 (time 1788740999.55 total 60.06 last 64.32) total: 4500 open/close in 68.02 seconds: 66.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 936 (time 1788741022.69 total 10.03 last 93.28) - /unlink 2034 (time 1788741032.70 total 20.05 last 109.63) - /unlink 2992 (time 1788741042.71 total 30.06 last 95.72) - /unlink 4173 (time 1788741052.71 total 40.06 last 118.06) total: 4500 /unlink in 43.92 seconds: 102.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 594 (time 1788741071.94 total 10.00 last 59.39) - open/close 1243 (time 1788741081.95 total 20.00 last 64.89) - open/close 1907 (time 1788741091.97 total 30.03 last 66.24) - open/close 2587 (time 1788741101.97 total 40.03 last 67.97) - open/close 3243 (time 1788741112.00 total 50.06 last 65.45) - open/close 3907 (time 1788741122.01 total 60.06 last 66.35) total: 4500 open/close in 67.93 seconds: 66.24 ops/second - /unlink 1460 (time 1788741144.68 total 10.00 last 145.97) - /unlink 3050 (time 1788741154.68 total 20.00 last 158.98) total: 4500 /unlink in 29.53 seconds: 152.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 895 (time 1788741179.78 total 10.01 last 89.44) - open/close 1711 (time 1788741189.79 total 20.01 last 81.53) - open/close 2487 (time 1788741199.80 total 30.02 last 77.54) - open/close 3310 (time 1788741209.81 total 40.03 last 82.23) - open/close 4088 (time 1788741219.81 total 50.03 last 77.79) total: 4500 open/close in 54.65 seconds: 82.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1593 (time 1788741238.68 total 10.00 last 159.27) - /unlink 3076 (time 1788741248.68 total 20.00 last 148.28) total: 4500 /unlink in 29.44 seconds: 152.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 761 (time 1788741274.29 total 10.01 last 76.03) - open/close 1663 (time 1788741284.30 total 20.02 last 90.11) - open/close 2434 (time 1788741294.31 total 30.03 last 77.00) - open/close 3312 (time 1788741304.33 total 40.05 last 87.65) - open/close 4207 (time 1788741314.34 total 50.06 last 89.41) total: 4500 open/close in 53.96 seconds: 83.40 ops/second - /unlink 1682 (time 1788741331.94 total 10.00 last 168.17) - /unlink 3095 (time 1788741341.94 total 20.00 last 141.29) - /unlink 4368 (time 1788741351.94 total 30.00 last 127.30) total: 4500 /unlink in 31.43 seconds: 143.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 682 (time 1788741369.01 total 10.00 last 68.18) - open/close 1409 (time 1788741379.01 total 20.01 last 72.67) - open/close 2025 (time 1788741389.02 total 30.01 last 61.58) - open/close 2556 (time 1788741399.03 total 40.03 last 53.01) - open/close 3098 (time 1788741409.05 total 50.04 last 54.13) - open/close 3659 (time 1788741419.05 total 60.05 last 56.07) - open/close 4378 (time 1788741429.07 total 70.06 last 71.77) total: 4500 open/close in 72.30 seconds: 62.24 ops/second - /unlink 1112 (time 1788741446.29 total 10.00 last 111.20) - /unlink 2006 (time 1788741456.30 total 20.01 last 89.29) - /unlink 3286 (time 1788741466.30 total 30.01 last 128.00) total: 4500 /unlink in 38.77 seconds: 116.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 780 (time 1788741490.41 total 10.01 last 77.91) - open/close 1404 (time 1788741500.42 total 20.02 last 62.34) - open/close 2001 (time 1788741510.42 total 30.03 last 59.68) - open/close 2544 (time 1788741520.46 total 40.06 last 54.10) - open/close 3119 (time 1788741530.52 total 50.12 last 57.14) - open/close 3866 (time 1788741540.53 total 60.14 last 74.60) - open/close 4468 (time 1788741550.54 total 70.15 last 60.14) total: 4500 open/close in 70.96 seconds: 63.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 972 (time 1788741568.49 total 10.01 last 97.08) - /unlink 1893 (time 1788741578.50 total 20.02 last 92.04) - /unlink 2753 (time 1788741588.50 total 30.02 last 85.99) - /unlink 3879 (time 1788741598.50 total 40.02 last 112.57) total: 4500 /unlink in 44.29 seconds: 101.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 726 (time 1788741617.85 total 10.00 last 72.58) - open/close 1409 (time 1788741627.87 total 20.02 last 68.18) - open/close 2243 (time 1788741637.87 total 30.02 last 83.40) - open/close 3053 (time 1788741647.88 total 40.03 last 80.95) - open/close 3670 (time 1788741657.89 total 50.04 last 61.61) - open/close 4369 (time 1788741667.91 total 60.06 last 69.79) total: 4500 open/close in 61.99 seconds: 72.59 ops/second - /unlink 1497 (time 1788741684.20 total 10.00 last 149.63) - /unlink 3018 (time 1788741694.20 total 20.01 last 152.04) - /unlink 4496 (time 1788741704.21 total 30.01 last 147.76) total: 4500 /unlink in 30.03 seconds: 149.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 791 (time 1788741718.03 total 10.01 last 79.06) - open/close 1595 (time 1788741728.04 total 20.01 last 80.36) - open/close 2351 (time 1788741738.04 total 30.01 last 75.58) - open/close 3176 (time 1788741748.05 total 40.02 last 82.45) - open/close 4032 (time 1788741758.05 total 50.03 last 85.54) total: 4500 open/close in 56.03 seconds: 80.31 ops/second - /unlink 1592 (time 1788741778.21 total 10.00 last 159.20) - /unlink 3130 (time 1788741788.21 total 20.00 last 153.73) total: 4500 /unlink in 28.80 seconds: 156.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 778 (time 1788741811.53 total 10.00 last 77.76) - open/close 1561 (time 1788741821.53 total 20.01 last 78.25) - open/close 2378 (time 1788741831.54 total 30.01 last 81.68) - open/close 3197 (time 1788741841.54 total 40.02 last 81.87) - open/close 4003 (time 1788741851.55 total 50.03 last 80.53) total: 4500 open/close in 56.04 seconds: 80.30 ops/second - /unlink 1514 (time 1788741871.31 total 10.00 last 151.37) - /unlink 2890 (time 1788741881.31 total 20.01 last 137.55) - /unlink 4376 (time 1788741891.31 total 30.01 last 148.60) total: 4500 /unlink in 31.24 seconds: 144.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 920 (time 1788741906.73 total 10.01 last 91.93) - open/close 1829 (time 1788741916.74 total 20.01 last 90.87) - open/close 2695 (time 1788741926.74 total 30.01 last 86.59) - open/close 3567 (time 1788741936.74 total 40.02 last 87.14) total: 4500 open/close in 49.15 seconds: 91.55 ops/second - /unlink 1880 (time 1788741958.57 total 10.00 last 187.95) - /unlink 3456 (time 1788741968.58 total 20.01 last 157.48) total: 4500 /unlink in 26.67 seconds: 168.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 979 (time 1788741988.81 total 10.00 last 97.87) - open/close 1965 (time 1788741998.82 total 20.02 last 98.48) - open/close 2897 (time 1788742008.83 total 30.02 last 93.13) - open/close 3804 (time 1788742018.84 total 40.03 last 90.64) total: 4500 open/close in 47.24 seconds: 95.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2023 (time 1788742038.66 total 10.00 last 202.29) - /unlink 3711 (time 1788742048.66 total 20.01 last 168.69) total: 4500 /unlink in 24.66 seconds: 182.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 816 (time 1788742066.84 total 10.00 last 81.56) - open/close 1652 (time 1788742076.85 total 20.01 last 83.56) - open/close 2520 (time 1788742086.85 total 30.01 last 86.76) - open/close 3295 (time 1788742096.85 total 40.01 last 77.50) - open/close 4041 (time 1788742106.86 total 50.02 last 74.54) total: 4500 open/close in 55.33 seconds: 81.33 ops/second - /unlink 1600 (time 1788742125.95 total 10.00 last 159.93) - /unlink 2815 (time 1788742135.98 total 20.04 last 121.08) - /unlink 4129 (time 1788742145.99 total 30.04 last 131.38) total: 4500 /unlink in 32.16 seconds: 139.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 821 (time 1788742161.67 total 10.02 last 81.93) - open/close 1490 (time 1788742171.69 total 20.05 last 66.74) - open/close 2089 (time 1788742181.75 total 30.11 last 59.54) - open/close 2725 (time 1788742191.76 total 40.11 last 63.58) - open/close 3478 (time 1788742201.76 total 50.11 last 75.26) - open/close 4212 (time 1788742211.77 total 60.12 last 73.35) total: 4500 open/close in 63.32 seconds: 71.07 ops/second - /unlink 1269 (time 1788742229.22 total 10.03 last 126.53) - /unlink 2128 (time 1788742239.23 total 20.04 last 85.82) - /unlink 3206 (time 1788742249.25 total 30.06 last 107.54) total: 4500 /unlink in 39.99 seconds: 112.52 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 94862 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg342-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg342-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg342-server stop mds service on oleg342-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg342-server stop mds service on oleg342-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg342-server unloading modules via unload_modules_local on: 'oleg342-server' oleg342-server: oleg342-server.virtnet: executing unload_modules_local oleg342-server: kmemleak disabled modules unloaded.