== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 19:19:33 (1789687173) Stopping clients: oleg212-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg212-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg212-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg212-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg212-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg212-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg212-server oleg212-server: oleg212-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg212-server' oleg212-server: oleg212-server.virtnet: executing load_modules_local oleg212-server: kmemleak disabled oleg212-server: Loading modules from /home/green/git/lustre-release/lustre oleg212-server: detected 4 online CPUs by sysfs oleg212-server: MODOPTS_LIBCFS= oleg212-server: Force libcfs to create 2 CPU partitions oleg212-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg212-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg212-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg212-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg212-client.virtnet: -o user_xattr,flock 192.168.202.112@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 746 (time 1789687404.28 total 10.00 last 74.58) - open/close 1472 (time 1789687414.28 total 20.00 last 72.60) - open/close 2098 (time 1789687424.28 total 30.00 last 62.60) - open/close 2835 (time 1789687434.29 total 40.02 last 73.59) - open/close 3359 (time 1789687444.30 total 50.02 last 52.38) - open/close 4012 (time 1789687454.31 total 60.03 last 65.23) total: 4500 open/close in 68.07 seconds: 66.11 ops/second - /unlink 1181 (time 1789687478.82 total 10.00 last 118.05) - /unlink 2358 (time 1789687488.82 total 20.01 last 117.69) - /unlink 3449 (time 1789687498.83 total 30.01 last 109.09) total: 4500 /unlink in 37.04 seconds: 121.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 729 (time 1789687521.00 total 10.02 last 72.74) - open/close 1381 (time 1789687531.00 total 20.02 last 65.19) - open/close 2013 (time 1789687541.02 total 30.04 last 63.08) - open/close 2741 (time 1789687551.03 total 40.05 last 72.76) - open/close 3369 (time 1789687561.04 total 50.05 last 62.75) - open/close 4065 (time 1789687571.04 total 60.05 last 69.60) total: 4500 open/close in 66.50 seconds: 67.67 ops/second - /unlink 1372 (time 1789687592.22 total 10.00 last 137.15) - /unlink 2672 (time 1789687602.23 total 20.01 last 129.97) - /unlink 4090 (time 1789687612.23 total 30.01 last 141.80) total: 4500 /unlink in 33.42 seconds: 134.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 652 (time 1789687632.89 total 10.01 last 65.14) - open/close 1307 (time 1789687642.90 total 20.02 last 65.45) - open/close 1940 (time 1789687652.91 total 30.02 last 63.25) - open/close 2673 (time 1789687662.92 total 40.04 last 73.19) - open/close 3316 (time 1789687672.93 total 50.05 last 64.24) - open/close 4006 (time 1789687682.95 total 60.07 last 68.88) total: 4500 open/close in 66.97 seconds: 67.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1216 (time 1789687705.27 total 10.00 last 121.57) - /unlink 2368 (time 1789687715.28 total 20.01 last 115.14) - /unlink 3556 (time 1789687725.28 total 30.01 last 118.76) total: 4500 /unlink in 39.01 seconds: 115.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 694 (time 1789687749.52 total 10.01 last 69.36) - open/close 1414 (time 1789687759.52 total 20.01 last 71.99) - open/close 2105 (time 1789687769.53 total 30.01 last 69.07) - open/close 2769 (time 1789687779.54 total 40.02 last 66.33) - open/close 3481 (time 1789687789.54 total 50.03 last 71.17) - open/close 4164 (time 1789687799.55 total 60.04 last 68.23) total: 4500 open/close in 65.01 seconds: 69.22 ops/second - /unlink 1429 (time 1789687819.86 total 10.00 last 142.89) - /unlink 2727 (time 1789687829.86 total 20.01 last 129.73) - /unlink 3997 (time 1789687839.87 total 30.01 last 126.92) total: 4500 /unlink in 34.06 seconds: 132.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 711 (time 1789687859.97 total 10.00 last 71.07) - open/close 1339 (time 1789687869.98 total 20.01 last 62.74) - open/close 2050 (time 1789687879.98 total 30.02 last 71.07) - open/close 2749 (time 1789687890.00 total 40.04 last 69.77) - open/close 3498 (time 1789687900.00 total 50.04 last 74.89) - open/close 4240 (time 1789687910.01 total 60.04 last 74.16) total: 4500 open/close in 63.73 seconds: 70.61 ops/second - /unlink 1339 (time 1789687928.52 total 10.01 last 133.73) - /unlink 2786 (time 1789687938.53 total 20.02 last 144.57) - /unlink 4028 (time 1789687948.54 total 30.03 last 124.13) total: 4500 /unlink in 34.13 seconds: 131.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 673 (time 1789687967.33 total 10.00 last 67.28) - open/close 1306 (time 1789687977.33 total 20.00 last 63.30) - open/close 1869 (time 1789687987.34 total 30.01 last 56.25) - open/close 2515 (time 1789687997.36 total 40.03 last 64.50) - open/close 3216 (time 1789688007.38 total 50.05 last 69.98) - open/close 3935 (time 1789688017.38 total 60.06 last 71.84) total: 4500 open/close in 69.05 seconds: 65.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1324 (time 1789688041.62 total 10.01 last 132.21) - /unlink 2393 (time 1789688051.62 total 20.02 last 106.90) - /unlink 3421 (time 1789688061.62 total 30.02 last 102.77) total: 4500 /unlink in 38.69 seconds: 116.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 626 (time 1789688085.93 total 10.01 last 62.52) - open/close 1271 (time 1789688095.94 total 20.02 last 64.45) - open/close 1979 (time 1789688105.94 total 30.02 last 70.80) - open/close 2656 (time 1789688115.95 total 40.03 last 67.64) - open/close 3234 (time 1789688125.95 total 50.04 last 57.76) - open/close 3879 (time 1789688135.96 total 60.04 last 64.46) - open/close 4343 (time 1789688145.98 total 70.06 last 46.32) total: 4500 open/close in 72.82 seconds: 61.79 ops/second - /unlink 1317 (time 1789688164.36 total 10.00 last 131.67) - /unlink 2434 (time 1789688174.36 total 20.00 last 111.69) - /unlink 3862 (time 1789688184.36 total 30.01 last 142.76) total: 4500 /unlink in 34.89 seconds: 128.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 583 (time 1789688204.39 total 10.01 last 58.26) - open/close 1332 (time 1789688214.39 total 20.01 last 74.90) - open/close 2070 (time 1789688224.40 total 30.02 last 73.72) - open/close 2949 (time 1789688234.42 total 40.04 last 87.69) - open/close 3713 (time 1789688244.43 total 50.05 last 76.35) total: 4500 open/close in 59.56 seconds: 75.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1598 (time 1789688269.04 total 10.00 last 159.76) - /unlink 3045 (time 1789688279.05 total 20.01 last 144.61) - /unlink 4378 (time 1789688289.06 total 30.02 last 133.14) total: 4500 /unlink in 31.21 seconds: 144.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 682 (time 1789688306.32 total 10.01 last 68.15) - open/close 1329 (time 1789688316.35 total 20.04 last 64.51) - open/close 2075 (time 1789688326.36 total 30.04 last 74.55) - open/close 2729 (time 1789688336.36 total 40.04 last 65.40) - open/close 3358 (time 1789688346.36 total 50.05 last 62.89) - open/close 4084 (time 1789688356.37 total 60.05 last 72.55) total: 4500 open/close in 66.45 seconds: 67.72 ops/second - /unlink 1408 (time 1789688378.08 total 10.02 last 140.54) - /unlink 2672 (time 1789688388.08 total 20.02 last 126.35) - /unlink 4087 (time 1789688398.09 total 30.03 last 141.37) total: 4500 /unlink in 33.35 seconds: 134.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 804 (time 1789688418.00 total 10.00 last 80.40) - open/close 1537 (time 1789688428.01 total 20.01 last 73.22) - open/close 2172 (time 1789688438.02 total 30.01 last 63.48) - open/close 2706 (time 1789688448.04 total 40.03 last 53.30) - open/close 3187 (time 1789688458.05 total 50.05 last 48.03) - open/close 3804 (time 1789688468.06 total 60.05 last 61.67) total: 4500 open/close in 69.85 seconds: 64.43 ops/second - /unlink 1461 (time 1789688493.00 total 10.00 last 146.10) - /unlink 2722 (time 1789688503.01 total 20.01 last 126.02) - /unlink 4020 (time 1789688513.01 total 30.01 last 129.76) total: 4500 /unlink in 33.85 seconds: 132.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 598 (time 1789688532.88 total 10.01 last 59.72) - open/close 1221 (time 1789688542.89 total 20.02 last 62.24) - open/close 1928 (time 1789688552.90 total 30.04 last 70.62) - open/close 2574 (time 1789688562.91 total 40.05 last 64.54) - open/close 3240 (time 1789688572.94 total 50.07 last 66.44) - open/close 3926 (time 1789688582.94 total 60.07 last 68.58) total: 4500 open/close in 68.41 seconds: 65.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1411 (time 1789688605.52 total 10.01 last 141.00) - /unlink 2622 (time 1789688615.53 total 20.02 last 120.99) - /unlink 3805 (time 1789688625.53 total 30.02 last 118.30) total: 4500 /unlink in 35.44 seconds: 126.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 630 (time 1789688647.15 total 10.01 last 62.94) - open/close 1321 (time 1789688657.15 total 20.01 last 69.09) - open/close 2150 (time 1789688667.15 total 30.02 last 82.85) - open/close 2809 (time 1789688677.16 total 40.02 last 65.87) - open/close 3531 (time 1789688687.16 total 50.02 last 72.19) - open/close 4180 (time 1789688697.17 total 60.03 last 64.83) total: 4500 open/close in 64.82 seconds: 69.42 ops/second - /unlink 1241 (time 1789688716.55 total 10.00 last 124.07) - /unlink 2367 (time 1789688726.55 total 20.00 last 112.58) - /unlink 3573 (time 1789688736.56 total 30.01 last 120.52) total: 4500 /unlink in 38.30 seconds: 117.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 410 (time 1789688761.66 total 10.02 last 40.90) - open/close 946 (time 1789688771.67 total 20.03 last 53.55) - open/close 1557 (time 1789688781.73 total 30.09 last 60.76) - open/close 2230 (time 1789688791.74 total 40.10 last 67.24) - open/close 2954 (time 1789688801.76 total 50.12 last 72.23) - open/close 3713 (time 1789688811.76 total 60.12 last 75.90) - open/close 4443 (time 1789688821.77 total 70.13 last 72.94) total: 4500 open/close in 71.13 seconds: 63.27 ops/second - /unlink 1341 (time 1789688837.25 total 10.00 last 134.06) - /unlink 2343 (time 1789688847.25 total 20.01 last 100.18) - /unlink 3534 (time 1789688857.26 total 30.01 last 119.07) total: 4500 /unlink in 39.49 seconds: 113.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 604 (time 1789688882.77 total 10.01 last 60.34) - open/close 1271 (time 1789688892.78 total 20.02 last 66.61) - open/close 1948 (time 1789688902.79 total 30.03 last 67.63) - open/close 2669 (time 1789688912.80 total 40.04 last 72.03) - open/close 3419 (time 1789688922.83 total 50.08 last 74.77) - open/close 4049 (time 1789688932.84 total 60.08 last 63.00) total: 4500 open/close in 66.95 seconds: 67.21 ops/second - /unlink 1376 (time 1789688955.37 total 10.00 last 137.55) - /unlink 2643 (time 1789688965.37 total 20.01 last 126.66) - /unlink 3706 (time 1789688975.37 total 30.01 last 106.26) total: 4500 /unlink in 37.02 seconds: 121.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 601 (time 1789688998.04 total 10.00 last 60.08) - open/close 1186 (time 1789689008.06 total 20.02 last 58.38) - open/close 1775 (time 1789689018.07 total 30.02 last 58.89) - open/close 2595 (time 1789689028.07 total 40.03 last 81.94) - open/close 3324 (time 1789689038.10 total 50.06 last 72.69) - open/close 4054 (time 1789689048.10 total 60.06 last 72.98) total: 4500 open/close in 66.32 seconds: 67.85 ops/second - /unlink 1436 (time 1789689069.67 total 10.01 last 143.44) - /unlink 2684 (time 1789689079.67 total 20.01 last 124.77) - /unlink 3952 (time 1789689089.68 total 30.02 last 126.74) total: 4500 /unlink in 34.10 seconds: 131.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 647 (time 1789689108.76 total 10.00 last 64.67) - open/close 1311 (time 1789689118.77 total 20.01 last 66.34) - open/close 1944 (time 1789689128.78 total 30.02 last 63.27) - open/close 2617 (time 1789689138.78 total 40.02 last 67.27) - open/close 3307 (time 1789689148.78 total 50.02 last 69.00) - open/close 4123 (time 1789689158.79 total 60.03 last 81.56) total: 4500 open/close in 64.37 seconds: 69.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1420 (time 1789689178.28 total 10.01 last 141.80) - /unlink 2547 (time 1789689188.29 total 20.02 last 112.66) - /unlink 4002 (time 1789689198.30 total 30.03 last 145.34) total: 4500 /unlink in 33.28 seconds: 135.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 738 (time 1789689217.43 total 10.01 last 73.70) - open/close 1328 (time 1789689227.45 total 20.03 last 58.91) - open/close 1956 (time 1789689237.45 total 30.03 last 62.77) - open/close 2679 (time 1789689247.48 total 40.06 last 72.11) - open/close 3441 (time 1789689257.48 total 50.06 last 76.17) - open/close 4090 (time 1789689267.50 total 60.08 last 64.77) total: 4500 open/close in 65.55 seconds: 68.65 ops/second - /unlink 1276 (time 1789689287.83 total 10.02 last 127.40) - /unlink 2706 (time 1789689297.83 total 20.02 last 142.88) - /unlink 4029 (time 1789689307.84 total 30.03 last 132.16) total: 4500 /unlink in 33.88 seconds: 132.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 684 (time 1789689327.37 total 10.01 last 68.35) - open/close 1368 (time 1789689337.37 total 20.01 last 68.40) - open/close 1963 (time 1789689347.39 total 30.03 last 59.36) - open/close 2609 (time 1789689357.39 total 40.03 last 64.59) - open/close 3275 (time 1789689367.40 total 50.04 last 66.54) - open/close 3855 (time 1789689377.42 total 60.06 last 57.90) - open/close 4360 (time 1789689387.44 total 70.08 last 50.39) total: 4500 open/close in 72.66 seconds: 61.94 ops/second - /unlink 1216 (time 1789689405.80 total 10.00 last 121.59) - /unlink 2593 (time 1789689415.81 total 20.01 last 137.57) - /unlink 4187 (time 1789689425.81 total 30.01 last 159.38) total: 4500 /unlink in 32.21 seconds: 139.71 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95046 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg212-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg212-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg212-server stop mds service on oleg212-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg212-server stop mds service on oleg212-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg212-server unloading modules via unload_modules_local on: 'oleg212-server' oleg212-server: oleg212-server.virtnet: executing unload_modules_local oleg212-server: kmemleak disabled modules unloaded.