== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:35:13 (1788669313) Stopping clients: oleg630-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg630-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg630-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg630-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg630-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg630-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg630-server oleg630-server: oleg630-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg630-server' oleg630-server: oleg630-server.virtnet: executing load_modules_local oleg630-server: kmemleak disabled oleg630-server: Loading modules from /home/green/git/lustre-release/lustre oleg630-server: detected 4 online CPUs by sysfs oleg630-server: MODOPTS_LIBCFS= oleg630-server: Force libcfs to create 2 CPU partitions oleg630-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg630-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg630-server: oleg630-server.virtnet: executing set_default_debug -1 all pdsh@oleg630-client: oleg630-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg630-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg630-server: oleg630-server.virtnet: executing set_default_debug -1 all pdsh@oleg630-client: oleg630-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg630-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg630-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg630-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg630-server: oleg630-server.virtnet: executing set_default_debug -1 all pdsh@oleg630-client: oleg630-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg630-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg630-client.virtnet: -o user_xattr,flock 192.168.206.130@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 781 (time 1788669498.36 total 10.00 last 78.09) - open/close 1541 (time 1788669508.39 total 20.02 last 75.82) - open/close 2274 (time 1788669518.40 total 30.04 last 73.22) - open/close 3047 (time 1788669528.40 total 40.04 last 77.26) - open/close 3765 (time 1788669538.42 total 50.06 last 71.67) - open/close 4398 (time 1788669548.43 total 60.07 last 63.25) total: 4500 open/close in 61.90 seconds: 72.70 ops/second - /unlink 1379 (time 1788669564.66 total 10.00 last 137.88) - /unlink 2739 (time 1788669574.67 total 20.00 last 135.95) - /unlink 4045 (time 1788669584.67 total 30.01 last 130.56) total: 4500 /unlink in 33.59 seconds: 133.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 664 (time 1788669603.58 total 10.00 last 66.39) - open/close 1282 (time 1788669613.60 total 20.02 last 61.67) - open/close 1990 (time 1788669623.60 total 30.02 last 70.78) - open/close 2713 (time 1788669633.62 total 40.04 last 72.17) - open/close 3389 (time 1788669643.63 total 50.05 last 67.52) - open/close 4132 (time 1788669653.64 total 60.06 last 74.24) total: 4500 open/close in 65.01 seconds: 69.22 ops/second - /unlink 1429 (time 1788669671.96 total 10.00 last 142.90) - /unlink 2766 (time 1788669681.97 total 20.01 last 133.55) - /unlink 4161 (time 1788669691.97 total 30.01 last 139.47) total: 4500 /unlink in 32.51 seconds: 138.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 760 (time 1788669708.38 total 10.00 last 76.00) - open/close 1481 (time 1788669718.38 total 20.00 last 72.09) - open/close 2224 (time 1788669728.39 total 30.00 last 74.29) - open/close 2975 (time 1788669738.39 total 40.01 last 75.04) - open/close 3747 (time 1788669748.40 total 50.02 last 77.17) total: 4500 open/close in 59.90 seconds: 75.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1539 (time 1788669771.51 total 10.01 last 153.81) - /unlink 2905 (time 1788669781.51 total 20.01 last 136.54) - /unlink 4351 (time 1788669791.51 total 30.01 last 144.56) total: 4500 /unlink in 31.37 seconds: 143.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 815 (time 1788669807.53 total 10.01 last 81.45) - open/close 1558 (time 1788669817.54 total 20.02 last 74.21) - open/close 2266 (time 1788669827.56 total 30.04 last 70.66) - open/close 2984 (time 1788669837.58 total 40.05 last 71.70) - open/close 3733 (time 1788669847.58 total 50.06 last 74.85) - open/close 4471 (time 1788669857.59 total 60.06 last 73.76) total: 4500 open/close in 60.41 seconds: 74.49 ops/second - /unlink 1433 (time 1788669871.51 total 10.01 last 143.19) - /unlink 2919 (time 1788669881.52 total 20.01 last 148.53) - /unlink 4405 (time 1788669891.53 total 30.02 last 148.51) total: 4500 /unlink in 30.76 seconds: 146.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 704 (time 1788669906.03 total 10.01 last 70.33) - open/close 1451 (time 1788669916.03 total 20.01 last 74.69) - open/close 2165 (time 1788669926.03 total 30.01 last 71.38) - open/close 2698 (time 1788669936.03 total 40.02 last 53.29) - open/close 3302 (time 1788669946.04 total 50.02 last 60.36) - open/close 3995 (time 1788669956.05 total 60.04 last 69.21) total: 4500 open/close in 67.33 seconds: 66.83 ops/second - /unlink 1362 (time 1788669978.26 total 10.00 last 136.17) - /unlink 2761 (time 1788669988.26 total 20.00 last 139.90) - /unlink 4055 (time 1788669998.27 total 30.01 last 129.35) total: 4500 /unlink in 33.90 seconds: 132.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 765 (time 1788670016.62 total 10.00 last 76.49) - open/close 1595 (time 1788670026.64 total 20.02 last 82.88) - open/close 2379 (time 1788670036.64 total 30.02 last 78.34) - open/close 3152 (time 1788670046.65 total 40.03 last 77.26) - open/close 3937 (time 1788670056.65 total 50.03 last 78.47) total: 4500 open/close in 57.59 seconds: 78.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1393 (time 1788670078.47 total 10.03 last 138.89) - /unlink 2713 (time 1788670088.47 total 20.04 last 131.89) - /unlink 4047 (time 1788670098.48 total 30.04 last 133.38) total: 4500 /unlink in 33.20 seconds: 135.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 733 (time 1788670115.64 total 10.01 last 73.25) - open/close 1467 (time 1788670125.64 total 20.01 last 73.37) - open/close 2198 (time 1788670135.65 total 30.02 last 73.02) - open/close 2932 (time 1788670145.66 total 40.03 last 73.34) - open/close 3717 (time 1788670155.66 total 50.03 last 78.48) - open/close 4498 (time 1788670165.67 total 60.04 last 78.05) total: 4500 open/close in 60.05 seconds: 74.94 ops/second - /unlink 1549 (time 1788670179.05 total 10.01 last 154.78) - /unlink 2991 (time 1788670189.05 total 20.01 last 144.14) - /unlink 4418 (time 1788670199.06 total 30.02 last 142.61) total: 4500 /unlink in 30.57 seconds: 147.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 757 (time 1788670213.42 total 10.00 last 75.67) - open/close 1497 (time 1788670223.43 total 20.01 last 73.96) - open/close 2260 (time 1788670233.43 total 30.01 last 76.28) - open/close 3036 (time 1788670243.44 total 40.02 last 77.54) - open/close 3731 (time 1788670253.47 total 50.05 last 69.26) total: 4500 open/close in 59.96 seconds: 75.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1543 (time 1788670277.70 total 10.01 last 154.19) - /unlink 2913 (time 1788670287.71 total 20.01 last 136.96) - /unlink 4379 (time 1788670297.71 total 30.02 last 146.50) total: 4500 /unlink in 31.02 seconds: 145.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 719 (time 1788670313.46 total 10.00 last 71.89) - open/close 1460 (time 1788670323.48 total 20.02 last 73.95) - open/close 2213 (time 1788670333.48 total 30.02 last 75.30) - open/close 2985 (time 1788670343.49 total 40.03 last 77.18) - open/close 3773 (time 1788670353.49 total 50.03 last 78.79) total: 4500 open/close in 59.61 seconds: 75.49 ops/second - /unlink 1443 (time 1788670376.95 total 10.00 last 144.28) - /unlink 2893 (time 1788670386.96 total 20.00 last 144.95) - /unlink 4373 (time 1788670396.96 total 30.01 last 147.97) total: 4500 /unlink in 30.87 seconds: 145.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 730 (time 1788670411.22 total 10.00 last 72.97) - open/close 1490 (time 1788670421.23 total 20.01 last 75.92) - open/close 2280 (time 1788670431.24 total 30.02 last 78.93) - open/close 3106 (time 1788670441.25 total 40.03 last 82.54) - open/close 3888 (time 1788670451.25 total 50.03 last 78.19) total: 4500 open/close in 57.44 seconds: 78.35 ops/second - /unlink 1730 (time 1788670471.99 total 10.00 last 172.98) - /unlink 3512 (time 1788670481.99 total 20.00 last 178.17) total: 4500 /unlink in 26.61 seconds: 169.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 756 (time 1788670502.52 total 10.01 last 75.53) - open/close 1513 (time 1788670512.53 total 20.01 last 75.67) - open/close 2269 (time 1788670522.53 total 30.02 last 75.55) - open/close 2992 (time 1788670532.54 total 40.02 last 72.27) - open/close 3754 (time 1788670542.54 total 50.03 last 76.17) total: 4500 open/close in 59.99 seconds: 75.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1522 (time 1788670566.24 total 10.01 last 152.08) - /unlink 2950 (time 1788670576.25 total 20.01 last 142.72) total: 4500 /unlink in 29.51 seconds: 152.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 842 (time 1788670599.07 total 10.01 last 84.14) - open/close 1612 (time 1788670609.08 total 20.02 last 76.94) - open/close 2409 (time 1788670619.09 total 30.03 last 79.60) - open/close 3211 (time 1788670629.10 total 40.04 last 80.14) - open/close 3959 (time 1788670639.10 total 50.04 last 74.79) total: 4500 open/close in 56.70 seconds: 79.37 ops/second - /unlink 1340 (time 1788670660.72 total 10.00 last 133.94) - /unlink 2733 (time 1788670670.72 total 20.00 last 139.30) - /unlink 3932 (time 1788670680.75 total 30.04 last 119.47) total: 4500 /unlink in 35.45 seconds: 126.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 645 (time 1788670701.89 total 10.00 last 64.49) - open/close 1269 (time 1788670711.90 total 20.01 last 62.35) - open/close 1975 (time 1788670721.91 total 30.01 last 70.56) - open/close 2640 (time 1788670731.91 total 40.02 last 66.48) - open/close 3345 (time 1788670741.92 total 50.03 last 70.44) - open/close 4113 (time 1788670751.93 total 60.04 last 76.72) total: 4500 open/close in 64.72 seconds: 69.53 ops/second - /unlink 1469 (time 1788670770.19 total 10.00 last 146.87) - /unlink 2764 (time 1788670780.19 total 20.01 last 129.45) - /unlink 4083 (time 1788670790.19 total 30.01 last 131.84) total: 4500 /unlink in 33.12 seconds: 135.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 786 (time 1788670807.37 total 10.00 last 78.58) - open/close 1548 (time 1788670817.37 total 20.00 last 76.19) - open/close 2282 (time 1788670827.39 total 30.02 last 73.31) - open/close 3049 (time 1788670837.39 total 40.02 last 76.67) - open/close 3818 (time 1788670847.39 total 50.03 last 76.86) total: 4500 open/close in 59.10 seconds: 76.14 ops/second - /unlink 1450 (time 1788670870.65 total 10.01 last 144.82) - /unlink 2807 (time 1788670880.65 total 20.01 last 135.70) - /unlink 4191 (time 1788670890.66 total 30.02 last 138.27) total: 4500 /unlink in 32.77 seconds: 137.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 727 (time 1788670908.17 total 10.01 last 72.62) - open/close 1484 (time 1788670918.19 total 20.03 last 75.56) - open/close 2205 (time 1788670928.20 total 30.04 last 72.00) - open/close 2933 (time 1788670938.21 total 40.05 last 72.78) - open/close 3664 (time 1788670948.22 total 50.06 last 73.01) - open/close 4333 (time 1788670958.22 total 60.07 last 66.85) total: 4500 open/close in 62.55 seconds: 71.94 ops/second - /unlink 1209 (time 1788670974.65 total 10.00 last 120.88) - /unlink 2595 (time 1788670984.65 total 20.01 last 138.53) - /unlink 4013 (time 1788670994.66 total 30.02 last 141.61) total: 4500 /unlink in 33.76 seconds: 133.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 737 (time 1788671012.67 total 10.02 last 73.55) - open/close 1485 (time 1788671022.67 total 20.03 last 74.75) - open/close 2242 (time 1788671032.70 total 30.05 last 75.53) - open/close 3027 (time 1788671042.70 total 40.06 last 78.45) - open/close 3746 (time 1788671052.72 total 50.07 last 71.79) total: 4500 open/close in 59.54 seconds: 75.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1513 (time 1788671075.43 total 10.00 last 151.25) - /unlink 3028 (time 1788671085.43 total 20.00 last 151.48) total: 4500 /unlink in 29.62 seconds: 151.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 785 (time 1788671109.22 total 10.01 last 78.44) - open/close 1609 (time 1788671119.23 total 20.02 last 82.32) - open/close 2404 (time 1788671129.24 total 30.03 last 79.43) - open/close 3140 (time 1788671139.24 total 40.03 last 73.57) - open/close 3897 (time 1788671149.25 total 50.04 last 75.64) total: 4500 open/close in 58.45 seconds: 76.99 ops/second - /unlink 1467 (time 1788671171.36 total 10.01 last 146.60) - /unlink 2837 (time 1788671181.37 total 20.02 last 136.88) - /unlink 4250 (time 1788671191.37 total 30.02 last 141.26) total: 4500 /unlink in 32.05 seconds: 140.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 672 (time 1788671208.62 total 10.00 last 67.19) - open/close 1418 (time 1788671218.62 total 20.00 last 74.60) - open/close 2109 (time 1788671228.64 total 30.02 last 68.99) - open/close 2797 (time 1788671238.67 total 40.05 last 68.59) - open/close 3498 (time 1788671248.67 total 50.05 last 70.08) - open/close 4172 (time 1788671258.68 total 60.05 last 67.38) total: 4500 open/close in 64.81 seconds: 69.43 ops/second - /unlink 1320 (time 1788671278.05 total 10.00 last 131.97) - /unlink 2585 (time 1788671288.07 total 20.02 last 126.27) - /unlink 3927 (time 1788671298.07 total 30.03 last 134.13) total: 4500 /unlink in 33.74 seconds: 133.38 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95088 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg630-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg630-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg630-server stop mds service on oleg630-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg630-server stop mds service on oleg630-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg630-server unloading modules via unload_modules_local on: 'oleg630-server' oleg630-server: oleg630-server.virtnet: executing unload_modules_local oleg630-server: kmemleak disabled modules unloaded.