== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 18:12:01 (1788646321) Stopping clients: oleg102-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg102-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg102-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg102-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg102-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg102-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg102-server oleg102-server: oleg102-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg102-server' oleg102-server: oleg102-server.virtnet: executing load_modules_local oleg102-server: kmemleak disabled oleg102-server: Loading modules from /home/green/git/lustre-release/lustre oleg102-server: detected 4 online CPUs by sysfs oleg102-server: MODOPTS_LIBCFS= oleg102-server: Force libcfs to create 2 CPU partitions oleg102-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg102-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg102-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg102-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg102-server: oleg102-server.virtnet: executing set_default_debug -1 all pdsh@oleg102-client: oleg102-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg102-client.virtnet: -o user_xattr,flock 192.168.201.102@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 752 (time 1788646559.01 total 10.02 last 75.07) - open/close 1446 (time 1788646569.03 total 20.03 last 69.31) - open/close 2131 (time 1788646579.03 total 30.03 last 68.48) - open/close 2864 (time 1788646589.03 total 40.03 last 73.30) - open/close 3617 (time 1788646599.04 total 50.04 last 75.24) - open/close 4412 (time 1788646609.04 total 60.04 last 79.49) total: 4500 open/close in 61.35 seconds: 73.35 ops/second - /unlink 1414 (time 1788646625.66 total 10.00 last 141.33) - /unlink 2709 (time 1788646635.67 total 20.01 last 129.42) - /unlink 3922 (time 1788646645.67 total 30.01 last 121.28) total: 4500 /unlink in 34.57 seconds: 130.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 692 (time 1788646666.30 total 10.01 last 69.16) - open/close 1312 (time 1788646676.31 total 20.01 last 61.95) - open/close 1971 (time 1788646686.32 total 30.03 last 65.82) - open/close 2627 (time 1788646696.34 total 40.05 last 65.48) - open/close 3269 (time 1788646706.39 total 50.10 last 63.85) - open/close 3877 (time 1788646716.41 total 60.12 last 60.71) total: 4500 open/close in 69.23 seconds: 65.00 ops/second - /unlink 1695 (time 1788646740.12 total 10.00 last 169.44) - /unlink 3330 (time 1788646750.12 total 20.00 last 163.49) total: 4500 /unlink in 29.16 seconds: 154.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 744 (time 1788646774.22 total 10.01 last 74.35) - open/close 1600 (time 1788646784.22 total 20.01 last 85.57) - open/close 2475 (time 1788646794.23 total 30.01 last 87.47) - open/close 3313 (time 1788646804.24 total 40.02 last 83.74) - open/close 4116 (time 1788646814.25 total 50.03 last 80.21) total: 4500 open/close in 55.40 seconds: 81.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1348 (time 1788646833.80 total 10.01 last 134.65) - /unlink 2697 (time 1788646843.82 total 20.02 last 134.75) - /unlink 4254 (time 1788646853.82 total 30.02 last 155.67) total: 4500 /unlink in 32.23 seconds: 139.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 886 (time 1788646871.17 total 10.00 last 88.56) - open/close 1603 (time 1788646881.18 total 20.01 last 71.68) - open/close 2374 (time 1788646891.18 total 30.01 last 77.06) - open/close 3023 (time 1788646901.18 total 40.02 last 64.88) - open/close 3780 (time 1788646911.20 total 50.03 last 75.59) - open/close 4491 (time 1788646921.21 total 60.04 last 71.05) total: 4500 open/close in 60.12 seconds: 74.85 ops/second - /unlink 1198 (time 1788646936.64 total 10.00 last 119.75) - /unlink 2350 (time 1788646946.64 total 20.01 last 115.15) - /unlink 3883 (time 1788646956.64 total 30.01 last 153.28) total: 4500 /unlink in 34.61 seconds: 130.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 818 (time 1788646975.46 total 10.01 last 81.76) - open/close 1660 (time 1788646985.46 total 20.01 last 84.17) - open/close 2476 (time 1788646995.47 total 30.02 last 81.52) - open/close 3247 (time 1788647005.48 total 40.03 last 77.02) - open/close 4108 (time 1788647015.49 total 50.04 last 86.04) total: 4500 open/close in 54.43 seconds: 82.67 ops/second - /unlink 1538 (time 1788647034.12 total 10.00 last 153.78) - /unlink 3116 (time 1788647044.12 total 20.00 last 157.76) total: 4500 /unlink in 29.28 seconds: 153.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 834 (time 1788647068.48 total 10.01 last 83.35) - open/close 1595 (time 1788647078.48 total 20.01 last 76.09) - open/close 2432 (time 1788647088.50 total 30.03 last 83.51) - open/close 3115 (time 1788647098.51 total 40.04 last 68.26) - open/close 3914 (time 1788647108.52 total 50.05 last 79.82) total: 4500 open/close in 59.84 seconds: 75.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1245 (time 1788647133.25 total 10.00 last 124.45) - /unlink 2631 (time 1788647143.26 total 20.01 last 138.49) - /unlink 4089 (time 1788647153.27 total 30.02 last 145.65) total: 4500 /unlink in 33.48 seconds: 134.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 634 (time 1788647172.46 total 10.01 last 63.36) - open/close 1301 (time 1788647182.46 total 20.01 last 66.68) - open/close 2048 (time 1788647192.47 total 30.01 last 74.67) - open/close 2915 (time 1788647202.47 total 40.02 last 86.65) - open/close 3738 (time 1788647212.48 total 50.03 last 82.24) total: 4500 open/close in 58.22 seconds: 77.29 ops/second - /unlink 1521 (time 1788647235.28 total 10.00 last 152.09) - /unlink 2894 (time 1788647245.28 total 20.00 last 137.27) - /unlink 4401 (time 1788647255.28 total 30.00 last 150.69) total: 4500 /unlink in 30.92 seconds: 145.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 813 (time 1788647270.90 total 10.01 last 81.22) - open/close 1636 (time 1788647280.90 total 20.01 last 82.27) - open/close 2246 (time 1788647290.91 total 30.02 last 60.95) - open/close 3048 (time 1788647300.91 total 40.02 last 80.17) - open/close 3852 (time 1788647310.91 total 50.02 last 80.40) total: 4500 open/close in 57.72 seconds: 77.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1364 (time 1788647333.43 total 10.00 last 136.35) - /unlink 2579 (time 1788647343.44 total 20.00 last 121.49) - /unlink 3940 (time 1788647353.44 total 30.01 last 136.08) total: 4500 /unlink in 34.26 seconds: 131.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 747 (time 1788647372.62 total 10.01 last 74.66) - open/close 1449 (time 1788647382.63 total 20.02 last 70.13) - open/close 2101 (time 1788647392.63 total 30.02 last 65.19) - open/close 2818 (time 1788647402.64 total 40.03 last 71.61) - open/close 3509 (time 1788647412.65 total 50.04 last 69.05) - open/close 4191 (time 1788647422.65 total 60.04 last 68.18) total: 4500 open/close in 64.75 seconds: 69.50 ops/second - /unlink 1323 (time 1788647442.25 total 10.00 last 132.25) - /unlink 2527 (time 1788647452.25 total 20.01 last 120.36) - /unlink 3738 (time 1788647462.26 total 30.01 last 121.05) total: 4500 /unlink in 36.18 seconds: 124.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 515 (time 1788647484.42 total 10.03 last 51.35) - open/close 1017 (time 1788647494.43 total 20.04 last 50.16) - open/close 1498 (time 1788647504.43 total 30.05 last 48.06) - open/close 2122 (time 1788647514.45 total 40.06 last 62.30) - open/close 2795 (time 1788647524.48 total 50.09 last 67.10) - open/close 3467 (time 1788647534.48 total 60.09 last 67.19) - open/close 4293 (time 1788647544.49 total 70.10 last 82.52) total: 4500 open/close in 72.75 seconds: 61.86 ops/second - /unlink 1346 (time 1788647561.89 total 10.00 last 134.59) - /unlink 2842 (time 1788647571.90 total 20.00 last 149.55) - /unlink 4233 (time 1788647581.91 total 30.01 last 138.96) total: 4500 /unlink in 32.16 seconds: 139.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 762 (time 1788647599.44 total 10.00 last 76.20) - open/close 1537 (time 1788647609.45 total 20.01 last 77.42) - open/close 2377 (time 1788647619.49 total 30.05 last 83.63) - open/close 3091 (time 1788647629.49 total 40.06 last 71.39) - open/close 3868 (time 1788647639.49 total 50.06 last 77.68) - open/close 4491 (time 1788647649.50 total 60.06 last 62.28) total: 4500 open/close in 60.15 seconds: 74.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1254 (time 1788647664.92 total 10.01 last 125.27) - /unlink 2447 (time 1788647674.92 total 20.01 last 119.26) - /unlink 3618 (time 1788647684.93 total 30.02 last 117.08) total: 4500 /unlink in 37.71 seconds: 119.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 611 (time 1788647708.54 total 10.01 last 61.05) - open/close 1203 (time 1788647718.56 total 20.03 last 59.09) - open/close 1798 (time 1788647728.56 total 30.03 last 59.49) - open/close 2590 (time 1788647738.56 total 40.03 last 79.17) - open/close 3246 (time 1788647748.58 total 50.05 last 65.47) - open/close 3860 (time 1788647758.60 total 60.07 last 61.29) - open/close 4361 (time 1788647768.61 total 70.08 last 50.07) total: 4500 open/close in 73.40 seconds: 61.31 ops/second - /unlink 1241 (time 1788647788.91 total 10.00 last 124.06) - /unlink 2753 (time 1788647798.92 total 20.01 last 151.14) - /unlink 4172 (time 1788647808.92 total 30.01 last 141.89) total: 4500 /unlink in 32.74 seconds: 137.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 625 (time 1788647827.86 total 10.01 last 62.46) - open/close 1407 (time 1788647837.89 total 20.03 last 78.01) - open/close 2256 (time 1788647847.89 total 30.04 last 84.85) - open/close 3085 (time 1788647857.90 total 40.04 last 82.86) - open/close 4008 (time 1788647867.90 total 50.05 last 92.23) total: 4500 open/close in 57.25 seconds: 78.60 ops/second - /unlink 979 (time 1788647890.66 total 10.00 last 97.89) - /unlink 2088 (time 1788647900.67 total 20.01 last 110.85) - /unlink 3594 (time 1788647910.68 total 30.02 last 150.41) total: 4500 /unlink in 36.25 seconds: 124.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 615 (time 1788647932.56 total 10.00 last 61.49) - open/close 1352 (time 1788647942.57 total 20.01 last 73.64) - open/close 2062 (time 1788647952.58 total 30.01 last 70.96) - open/close 2843 (time 1788647962.58 total 40.02 last 78.05) - open/close 3395 (time 1788647972.58 total 50.02 last 55.19) - open/close 3974 (time 1788647982.60 total 60.03 last 57.84) total: 4500 open/close in 68.66 seconds: 65.54 ops/second - /unlink 1246 (time 1788648006.50 total 10.03 last 124.23) - /unlink 2319 (time 1788648016.51 total 20.03 last 107.26) - /unlink 3520 (time 1788648026.51 total 30.04 last 120.06) total: 4500 /unlink in 38.25 seconds: 117.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 739 (time 1788648050.01 total 10.01 last 73.86) - open/close 1526 (time 1788648060.01 total 20.01 last 78.67) - open/close 2278 (time 1788648070.02 total 30.01 last 75.16) - open/close 2989 (time 1788648080.03 total 40.02 last 71.04) - open/close 3454 (time 1788648090.04 total 50.03 last 46.45) - open/close 4113 (time 1788648100.04 total 60.04 last 65.88) total: 4500 open/close in 65.37 seconds: 68.84 ops/second - /unlink 1387 (time 1788648120.55 total 10.00 last 138.66) - /unlink 2684 (time 1788648130.55 total 20.00 last 129.67) - /unlink 4145 (time 1788648140.56 total 30.01 last 145.99) total: 4500 /unlink in 32.22 seconds: 139.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 731 (time 1788648158.67 total 10.00 last 73.08) - open/close 1327 (time 1788648168.67 total 20.01 last 59.58) - open/close 2125 (time 1788648178.67 total 30.01 last 79.77) - open/close 2882 (time 1788648188.68 total 40.02 last 75.63) - open/close 3707 (time 1788648198.69 total 50.03 last 82.44) - open/close 4473 (time 1788648208.70 total 60.04 last 76.50) total: 4500 open/close in 60.44 seconds: 74.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1350 (time 1788648223.65 total 10.00 last 134.96) - /unlink 2787 (time 1788648233.65 total 20.00 last 143.69) - /unlink 4093 (time 1788648243.66 total 30.01 last 130.56) total: 4500 /unlink in 32.67 seconds: 137.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 522 (time 1788648261.82 total 10.02 last 52.12) - open/close 1125 (time 1788648271.82 total 20.02 last 60.28) - open/close 1677 (time 1788648281.85 total 30.05 last 55.02) - open/close 2136 (time 1788648291.87 total 40.06 last 45.85) - open/close 3004 (time 1788648301.87 total 50.07 last 86.72) - open/close 3905 (time 1788648311.88 total 60.08 last 90.07) total: 4500 open/close in 67.02 seconds: 67.14 ops/second - /unlink 1364 (time 1788648333.07 total 10.00 last 136.35) - /unlink 2682 (time 1788648343.07 total 20.01 last 131.77) - /unlink 4165 (time 1788648353.07 total 30.01 last 148.24) total: 4500 /unlink in 33.35 seconds: 134.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 884 (time 1788648372.08 total 10.01 last 88.27) - open/close 1605 (time 1788648382.09 total 20.02 last 72.03) - open/close 2351 (time 1788648392.11 total 30.04 last 74.48) - open/close 3150 (time 1788648402.12 total 40.05 last 79.83) - open/close 3843 (time 1788648412.14 total 50.07 last 69.19) total: 4500 open/close in 59.53 seconds: 75.59 ops/second - /unlink 1409 (time 1788648436.15 total 10.00 last 140.84) - /unlink 2764 (time 1788648446.15 total 20.00 last 135.50) - /unlink 4219 (time 1788648456.16 total 30.01 last 145.46) total: 4500 /unlink in 32.06 seconds: 140.37 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95245 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg102-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg102-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg102-server stop mds service on oleg102-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg102-server stop mds service on oleg102-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg102-server unloading modules via unload_modules_local on: 'oleg102-server' oleg102-server: oleg102-server.virtnet: executing unload_modules_local oleg102-server: kmemleak disabled modules unloaded.