== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:28:54 (1788920934) Stopping clients: oleg252-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg252-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg252-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg252-server oleg252-server: oleg252-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing load_modules_local oleg252-server: kmemleak disabled oleg252-server: Loading modules from /home/green/git/lustre-release/lustre oleg252-server: detected 4 online CPUs by sysfs oleg252-server: MODOPTS_LIBCFS= oleg252-server: Force libcfs to create 2 CPU partitions oleg252-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg252-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg252-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock 192.168.202.152@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 910 (time 1788921071.15 total 10.00 last 90.98) - open/close 1895 (time 1788921081.15 total 20.00 last 98.50) - open/close 2777 (time 1788921091.16 total 30.01 last 88.11) - open/close 3682 (time 1788921101.18 total 40.04 last 90.27) total: 4500 open/close in 49.33 seconds: 91.21 ops/second - /unlink 1283 (time 1788921124.34 total 10.01 last 128.17) - /unlink 2897 (time 1788921134.35 total 20.02 last 161.26) total: 4500 /unlink in 29.43 seconds: 152.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 915 (time 1788921157.90 total 10.00 last 91.50) - open/close 1688 (time 1788921167.91 total 20.01 last 77.23) - open/close 2554 (time 1788921177.92 total 30.01 last 86.58) - open/close 3342 (time 1788921187.92 total 40.02 last 78.75) - open/close 4135 (time 1788921197.93 total 50.03 last 79.23) total: 4500 open/close in 54.96 seconds: 81.87 ops/second - /unlink 1121 (time 1788921217.38 total 10.00 last 112.10) - /unlink 2434 (time 1788921227.38 total 20.00 last 131.28) - /unlink 3670 (time 1788921237.38 total 30.00 last 123.58) total: 4500 /unlink in 37.32 seconds: 120.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 771 (time 1788921259.85 total 10.01 last 77.06) - open/close 1544 (time 1788921269.87 total 20.02 last 77.17) - open/close 2357 (time 1788921279.87 total 30.03 last 81.27) - open/close 3226 (time 1788921289.88 total 40.03 last 86.84) - open/close 4208 (time 1788921299.89 total 50.04 last 98.09) total: 4500 open/close in 52.20 seconds: 86.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1541 (time 1788921315.80 total 10.00 last 154.08) - /unlink 3258 (time 1788921325.80 total 20.00 last 171.65) total: 4500 /unlink in 27.86 seconds: 161.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 895 (time 1788921347.15 total 10.01 last 89.45) - open/close 1831 (time 1788921357.17 total 20.02 last 93.44) - open/close 2819 (time 1788921367.17 total 30.03 last 98.77) - open/close 3708 (time 1788921377.18 total 40.03 last 88.83) - open/close 4498 (time 1788921387.18 total 50.04 last 78.95) total: 4500 open/close in 50.05 seconds: 89.92 ops/second - /unlink 1859 (time 1788921400.58 total 10.00 last 185.83) - /unlink 3723 (time 1788921410.58 total 20.01 last 186.35) total: 4500 /unlink in 24.55 seconds: 183.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 882 (time 1788921428.96 total 10.00 last 88.17) - open/close 1777 (time 1788921438.97 total 20.02 last 89.39) - open/close 2653 (time 1788921448.99 total 30.03 last 87.49) - open/close 3485 (time 1788921459.00 total 40.04 last 83.13) - open/close 4431 (time 1788921469.00 total 50.04 last 94.56) total: 4500 open/close in 50.69 seconds: 88.77 ops/second - /unlink 1676 (time 1788921482.74 total 10.00 last 167.60) - /unlink 3320 (time 1788921492.75 total 20.00 last 164.35) total: 4500 /unlink in 26.90 seconds: 167.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1040 (time 1788921513.27 total 10.01 last 103.93) - open/close 1997 (time 1788921523.28 total 20.02 last 95.59) - open/close 2929 (time 1788921533.30 total 30.04 last 93.02) - open/close 3773 (time 1788921543.30 total 40.04 last 84.38) total: 4500 open/close in 49.30 seconds: 91.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1199 (time 1788921567.23 total 10.00 last 119.90) - /unlink 2129 (time 1788921577.25 total 20.02 last 92.83) - /unlink 2814 (time 1788921587.26 total 30.03 last 68.43) - /unlink 3873 (time 1788921597.26 total 40.03 last 105.84) total: 4500 /unlink in 46.49 seconds: 96.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 729 (time 1788921620.34 total 10.00 last 72.89) - open/close 1558 (time 1788921630.36 total 20.02 last 82.72) - open/close 2383 (time 1788921640.38 total 30.04 last 82.35) - open/close 3011 (time 1788921650.40 total 40.06 last 62.68) - open/close 3708 (time 1788921660.42 total 50.08 last 69.55) - open/close 4450 (time 1788921670.42 total 60.09 last 74.16) total: 4500 open/close in 60.69 seconds: 74.14 ops/second - /unlink 1650 (time 1788921685.00 total 10.00 last 164.96) - /unlink 3235 (time 1788921695.00 total 20.00 last 158.49) total: 4500 /unlink in 29.19 seconds: 154.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 760 (time 1788921718.52 total 10.01 last 75.89) - open/close 1509 (time 1788921728.52 total 20.01 last 74.89) - open/close 2354 (time 1788921738.53 total 30.02 last 84.44) - open/close 3063 (time 1788921748.55 total 40.04 last 70.77) - open/close 3942 (time 1788921758.56 total 50.05 last 87.79) total: 4500 open/close in 55.77 seconds: 80.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1609 (time 1788921778.62 total 10.02 last 160.61) - /unlink 3212 (time 1788921788.63 total 20.03 last 160.17) - /unlink 4415 (time 1788921798.63 total 30.03 last 120.28) total: 4500 /unlink in 30.65 seconds: 146.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 690 (time 1788921814.25 total 10.00 last 69.00) - open/close 1371 (time 1788921824.27 total 20.01 last 68.02) - open/close 1992 (time 1788921834.27 total 30.02 last 62.08) - open/close 2724 (time 1788921844.28 total 40.02 last 73.13) - open/close 3624 (time 1788921854.29 total 50.04 last 89.89) - open/close 4251 (time 1788921864.30 total 60.04 last 62.66) total: 4500 open/close in 63.62 seconds: 70.73 ops/second - /unlink 1393 (time 1788921882.13 total 10.00 last 139.27) - /unlink 2828 (time 1788921892.13 total 20.00 last 143.47) - /unlink 4300 (time 1788921902.13 total 30.01 last 147.15) total: 4500 /unlink in 31.37 seconds: 143.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 802 (time 1788921917.83 total 10.02 last 80.06) - open/close 1403 (time 1788921927.84 total 20.03 last 60.00) - open/close 2178 (time 1788921937.85 total 30.04 last 77.43) - open/close 3005 (time 1788921947.86 total 40.05 last 82.65) - open/close 3843 (time 1788921957.87 total 50.06 last 83.74) total: 4500 open/close in 57.90 seconds: 77.72 ops/second - /unlink 1408 (time 1788921980.05 total 10.01 last 140.61) - /unlink 2775 (time 1788921990.05 total 20.01 last 136.69) - /unlink 4103 (time 1788922000.06 total 30.02 last 132.75) total: 4500 /unlink in 32.94 seconds: 136.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 749 (time 1788922017.95 total 10.01 last 74.83) - open/close 1606 (time 1788922027.95 total 20.01 last 85.69) - open/close 2297 (time 1788922037.96 total 30.01 last 69.10) - open/close 2978 (time 1788922047.96 total 40.01 last 68.09) - open/close 3793 (time 1788922057.97 total 50.03 last 81.38) - open/close 4422 (time 1788922067.98 total 60.03 last 62.85) total: 4500 open/close in 61.87 seconds: 72.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1048 (time 1788922086.56 total 10.02 last 104.57) - /unlink 2358 (time 1788922096.57 total 20.03 last 130.87) - /unlink 3617 (time 1788922106.57 total 30.04 last 125.86) total: 4500 /unlink in 35.27 seconds: 127.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 850 (time 1788922125.91 total 10.01 last 84.95) - open/close 1402 (time 1788922135.92 total 20.01 last 55.16) - open/close 1853 (time 1788922145.94 total 30.04 last 45.00) - open/close 2626 (time 1788922155.95 total 40.05 last 77.17) - open/close 3183 (time 1788922165.99 total 50.09 last 55.50) - open/close 4040 (time 1788922176.00 total 60.09 last 85.65) total: 4500 open/close in 65.68 seconds: 68.51 ops/second - /unlink 1345 (time 1788922195.61 total 10.00 last 134.50) - /unlink 2590 (time 1788922205.62 total 20.01 last 124.38) - /unlink 4102 (time 1788922215.62 total 30.01 last 151.17) total: 4500 /unlink in 32.75 seconds: 137.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 777 (time 1788922233.59 total 10.00 last 77.68) - open/close 1597 (time 1788922243.62 total 20.03 last 81.78) - open/close 2229 (time 1788922253.63 total 30.04 last 63.13) - open/close 2890 (time 1788922263.63 total 40.04 last 66.10) - open/close 3629 (time 1788922273.63 total 50.05 last 73.86) - open/close 4393 (time 1788922283.64 total 60.06 last 76.34) total: 4500 open/close in 61.09 seconds: 73.66 ops/second - /unlink 1515 (time 1788922298.40 total 10.00 last 151.49) - /unlink 2855 (time 1788922308.40 total 20.00 last 133.98) - /unlink 4241 (time 1788922318.40 total 30.01 last 138.55) total: 4500 /unlink in 32.07 seconds: 140.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 671 (time 1788922336.04 total 10.00 last 67.07) - open/close 1647 (time 1788922346.05 total 20.02 last 97.49) - open/close 2441 (time 1788922356.06 total 30.03 last 79.32) - open/close 3302 (time 1788922366.06 total 40.03 last 86.06) - open/close 3942 (time 1788922376.07 total 50.04 last 63.97) total: 4500 open/close in 57.50 seconds: 78.27 ops/second - /unlink 1284 (time 1788922398.44 total 10.01 last 128.31) - /unlink 2642 (time 1788922408.44 total 20.01 last 135.77) - /unlink 3972 (time 1788922418.45 total 30.01 last 132.93) total: 4500 /unlink in 33.87 seconds: 132.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 686 (time 1788922437.29 total 10.03 last 68.42) - open/close 1420 (time 1788922447.29 total 20.03 last 73.37) - open/close 2133 (time 1788922457.30 total 30.04 last 71.26) - open/close 2863 (time 1788922467.30 total 40.04 last 72.99) - open/close 3616 (time 1788922477.31 total 50.04 last 75.25) - open/close 4285 (time 1788922487.31 total 60.05 last 66.86) total: 4500 open/close in 62.88 seconds: 71.56 ops/second - /unlink 1220 (time 1788922505.70 total 10.00 last 121.99) - /unlink 2369 (time 1788922515.71 total 20.00 last 114.86) - /unlink 3434 (time 1788922525.71 total 30.01 last 106.43) total: 4500 /unlink in 38.06 seconds: 118.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 786 (time 1788922548.25 total 10.00 last 78.59) - open/close 1390 (time 1788922558.26 total 20.01 last 60.33) - open/close 2042 (time 1788922568.27 total 30.03 last 65.12) - open/close 2684 (time 1788922578.28 total 40.03 last 64.19) - open/close 3431 (time 1788922588.28 total 50.03 last 74.66) - open/close 4224 (time 1788922598.32 total 60.07 last 79.02) total: 4500 open/close in 63.83 seconds: 70.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1417 (time 1788922617.20 total 10.00 last 141.67) - /unlink 2684 (time 1788922627.21 total 20.01 last 126.58) - /unlink 3922 (time 1788922637.21 total 30.01 last 123.79) total: 4500 /unlink in 34.65 seconds: 129.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 769 (time 1788922657.80 total 10.01 last 76.83) - open/close 1584 (time 1788922667.81 total 20.02 last 81.43) - open/close 2410 (time 1788922677.82 total 30.03 last 82.50) - open/close 3289 (time 1788922687.82 total 40.03 last 87.88) - open/close 4083 (time 1788922697.84 total 50.05 last 79.26) total: 4500 open/close in 55.69 seconds: 80.80 ops/second - /unlink 1369 (time 1788922719.58 total 10.00 last 136.89) - /unlink 2660 (time 1788922729.58 total 20.00 last 129.08) - /unlink 4087 (time 1788922739.58 total 30.01 last 142.65) total: 4500 /unlink in 32.68 seconds: 137.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 770 (time 1788922757.34 total 10.01 last 76.94) - open/close 1498 (time 1788922767.35 total 20.01 last 72.76) - open/close 2230 (time 1788922777.35 total 30.01 last 73.19) - open/close 2991 (time 1788922787.36 total 40.03 last 75.99) - open/close 3635 (time 1788922797.36 total 50.03 last 64.39) - open/close 4117 (time 1788922807.37 total 60.03 last 48.19) total: 4500 open/close in 69.45 seconds: 64.80 ops/second - /unlink 1153 (time 1788922834.02 total 10.00 last 115.25) - /unlink 2445 (time 1788922844.03 total 20.02 last 128.99) - /unlink 3826 (time 1788922854.04 total 30.03 last 138.00) total: 4500 /unlink in 34.99 seconds: 128.59 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 75350 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg252-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server unloading modules via unload_modules_local on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing unload_modules_local oleg252-server: kmemleak disabled modules unloaded.