== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 19:43:38 (1788392618) Stopping clients: oleg243-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg243-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg243-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg243-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg243-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg243-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg243-server oleg243-server: oleg243-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg243-server' oleg243-server: oleg243-server.virtnet: executing load_modules_local oleg243-server: kmemleak disabled oleg243-server: Loading modules from /home/green/git/lustre-release/lustre oleg243-server: detected 4 online CPUs by sysfs oleg243-server: MODOPTS_LIBCFS= oleg243-server: Force libcfs to create 2 CPU partitions oleg243-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg243-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug -1 all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg243-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg243-server: oleg243-server.virtnet: executing set_default_debug -1 all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg243-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg243-server: oleg243-server.virtnet: executing set_default_debug -1 all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1580 (time 1788392716.55 total 10.01 last 157.83) - open/close 3074 (time 1788392726.55 total 20.01 last 149.39) total: 4500 open/close in 29.89 seconds: 150.56 ops/second - /unlink 3239 (time 1788392748.26 total 10.00 last 323.89) total: 4500 /unlink in 14.83 seconds: 303.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1619 (time 1788392765.25 total 10.00 last 161.89) - open/close 3179 (time 1788392775.26 total 20.00 last 155.98) - open/close 4052 (time 1788392785.29 total 30.04 last 86.97) total: 4500 open/close in 36.56 seconds: 123.07 ops/second - /unlink 1886 (time 1788392805.85 total 10.00 last 188.55) - /unlink 3250 (time 1788392815.85 total 20.01 last 136.36) total: 4500 /unlink in 29.30 seconds: 153.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 739 (time 1788392840.32 total 10.01 last 73.83) - open/close 1532 (time 1788392850.32 total 20.02 last 79.25) - open/close 2322 (time 1788392860.33 total 30.03 last 78.92) - open/close 3258 (time 1788392870.35 total 40.04 last 93.49) - open/close 4143 (time 1788392880.35 total 50.04 last 88.50) total: 4500 open/close in 53.67 seconds: 83.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2233 (time 1788392897.66 total 10.00 last 223.26) - /unlink 4392 (time 1788392907.66 total 20.00 last 215.88) total: 4500 /unlink in 20.82 seconds: 216.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 994 (time 1788392921.74 total 10.00 last 99.37) - open/close 2091 (time 1788392931.75 total 20.02 last 109.56) - open/close 3180 (time 1788392941.76 total 30.02 last 108.88) - open/close 4209 (time 1788392951.77 total 40.03 last 102.79) total: 4500 open/close in 42.48 seconds: 105.92 ops/second - /unlink 2279 (time 1788392966.80 total 10.00 last 227.89) - /unlink 4463 (time 1788392976.81 total 20.00 last 218.32) total: 4500 /unlink in 20.17 seconds: 223.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1037 (time 1788392989.98 total 10.01 last 103.61) - open/close 2100 (time 1788392999.99 total 20.01 last 106.25) - open/close 3281 (time 1788393009.99 total 30.01 last 118.08) - open/close 4361 (time 1788393020.00 total 40.02 last 107.92) total: 4500 open/close in 41.41 seconds: 108.66 ops/second - /unlink 2117 (time 1788393034.08 total 10.00 last 211.61) - /unlink 4470 (time 1788393044.08 total 20.00 last 235.29) total: 4500 /unlink in 20.16 seconds: 223.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 741 (time 1788393056.81 total 10.01 last 74.00) - open/close 1255 (time 1788393066.85 total 20.05 last 51.20) - open/close 1911 (time 1788393076.86 total 30.06 last 65.59) - open/close 2461 (time 1788393086.87 total 40.07 last 54.94) - open/close 2952 (time 1788393096.88 total 50.08 last 49.03) - open/close 3437 (time 1788393106.89 total 60.09 last 48.44) - open/close 3851 (time 1788393116.92 total 70.12 last 41.30) - open/close 4351 (time 1788393126.96 total 80.16 last 49.80) total: 4500 open/close in 82.17 seconds: 54.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1488 (time 1788393146.92 total 10.00 last 148.76) - /unlink 2904 (time 1788393156.92 total 20.01 last 141.55) - /unlink 4373 (time 1788393166.92 total 30.01 last 146.87) total: 4500 /unlink in 31.03 seconds: 145.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 596 (time 1788393183.61 total 10.02 last 59.47) - open/close 1524 (time 1788393193.62 total 20.03 last 92.76) - open/close 2334 (time 1788393203.63 total 30.04 last 80.92) - open/close 3175 (time 1788393213.63 total 40.04 last 84.05) - open/close 4193 (time 1788393223.64 total 50.05 last 101.72) total: 4500 open/close in 52.87 seconds: 85.12 ops/second - /unlink 1741 (time 1788393240.50 total 10.01 last 173.99) - /unlink 3162 (time 1788393250.50 total 20.01 last 142.08) total: 4500 /unlink in 29.01 seconds: 155.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 835 (time 1788393273.88 total 10.01 last 83.45) - open/close 1587 (time 1788393283.88 total 20.01 last 75.18) - open/close 2365 (time 1788393293.88 total 30.01 last 77.79) - open/close 2973 (time 1788393303.89 total 40.02 last 60.75) - open/close 3566 (time 1788393313.89 total 50.02 last 59.30) - open/close 4196 (time 1788393323.92 total 60.04 last 62.85) total: 4500 open/close in 64.26 seconds: 70.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1394 (time 1788393343.35 total 10.01 last 139.23) - /unlink 2651 (time 1788393353.35 total 20.02 last 125.66) - /unlink 3796 (time 1788393363.35 total 30.02 last 114.47) total: 4500 /unlink in 36.09 seconds: 124.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 795 (time 1788393385.30 total 10.00 last 79.48) - open/close 1631 (time 1788393395.30 total 20.00 last 83.59) - open/close 2475 (time 1788393405.31 total 30.01 last 84.32) - open/close 3286 (time 1788393415.31 total 40.02 last 81.04) - open/close 4170 (time 1788393425.32 total 50.03 last 88.32) total: 4500 open/close in 53.79 seconds: 83.66 ops/second - /unlink 1589 (time 1788393443.72 total 10.00 last 158.90) - /unlink 3158 (time 1788393453.72 total 20.00 last 156.88) total: 4500 /unlink in 28.77 seconds: 156.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 664 (time 1788393478.14 total 10.01 last 66.36) - open/close 1305 (time 1788393488.17 total 20.04 last 63.92) - open/close 1965 (time 1788393498.17 total 30.04 last 65.94) - open/close 2634 (time 1788393508.18 total 40.05 last 66.84) - open/close 3433 (time 1788393518.19 total 50.06 last 79.85) - open/close 4238 (time 1788393528.20 total 60.07 last 80.45) total: 4500 open/close in 62.88 seconds: 71.57 ops/second - /unlink 1396 (time 1788393547.64 total 10.00 last 139.55) - /unlink 2803 (time 1788393557.65 total 20.01 last 140.61) - /unlink 4235 (time 1788393567.65 total 30.01 last 143.13) total: 4500 /unlink in 31.95 seconds: 140.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 719 (time 1788393585.85 total 10.00 last 71.89) - open/close 1541 (time 1788393595.88 total 20.02 last 82.01) - open/close 2313 (time 1788393605.89 total 30.03 last 77.14) - open/close 3088 (time 1788393615.90 total 40.05 last 77.37) - open/close 3955 (time 1788393625.91 total 50.05 last 86.66) total: 4500 open/close in 56.38 seconds: 79.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1749 (time 1788393647.08 total 10.01 last 174.77) - /unlink 3157 (time 1788393657.08 total 20.01 last 140.76) total: 4500 /unlink in 30.00 seconds: 149.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 712 (time 1788393681.36 total 10.00 last 71.19) - open/close 1467 (time 1788393691.37 total 20.01 last 75.40) - open/close 2219 (time 1788393701.37 total 30.02 last 75.19) - open/close 3012 (time 1788393711.37 total 40.02 last 79.27) - open/close 3795 (time 1788393721.38 total 50.02 last 78.26) total: 4500 open/close in 58.80 seconds: 76.53 ops/second - /unlink 1421 (time 1788393745.31 total 10.00 last 142.08) - /unlink 2741 (time 1788393755.31 total 20.01 last 131.91) - /unlink 3879 (time 1788393765.32 total 30.02 last 113.72) total: 4500 /unlink in 34.30 seconds: 131.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 689 (time 1788393784.84 total 10.02 last 68.74) - open/close 1421 (time 1788393794.84 total 20.02 last 73.19) - open/close 2007 (time 1788393804.85 total 30.04 last 58.52) - open/close 2719 (time 1788393814.86 total 40.04 last 71.16) - open/close 3514 (time 1788393824.87 total 50.05 last 79.44) - open/close 4225 (time 1788393834.87 total 60.06 last 71.06) total: 4500 open/close in 63.50 seconds: 70.87 ops/second - /unlink 1415 (time 1788393853.45 total 10.00 last 141.49) - /unlink 2940 (time 1788393863.45 total 20.00 last 152.48) total: 4500 /unlink in 29.12 seconds: 154.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 580 (time 1788393887.99 total 10.01 last 57.96) - open/close 1191 (time 1788393898.00 total 20.01 last 61.07) - open/close 1959 (time 1788393908.01 total 30.03 last 76.69) - open/close 2670 (time 1788393918.03 total 40.04 last 71.01) - open/close 3396 (time 1788393928.03 total 50.04 last 72.57) - open/close 3994 (time 1788393938.04 total 60.05 last 59.76) total: 4500 open/close in 67.70 seconds: 66.47 ops/second - /unlink 1071 (time 1788393961.80 total 10.01 last 107.03) - /unlink 2257 (time 1788393971.82 total 20.03 last 118.34) - /unlink 3172 (time 1788393981.83 total 30.03 last 91.45) - /unlink 4490 (time 1788393991.83 total 40.04 last 131.77) total: 4500 /unlink in 40.08 seconds: 112.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 944 (time 1788394006.58 total 10.00 last 94.39) - open/close 1876 (time 1788394016.60 total 20.02 last 93.04) - open/close 2742 (time 1788394026.61 total 30.03 last 86.53) - open/close 3619 (time 1788394036.61 total 40.03 last 87.70) total: 4500 open/close in 49.78 seconds: 90.40 ops/second - /unlink 1690 (time 1788394060.99 total 10.00 last 168.99) - /unlink 3217 (time 1788394071.00 total 20.01 last 152.60) total: 4500 /unlink in 28.92 seconds: 155.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 676 (time 1788394094.63 total 10.01 last 67.54) - open/close 1299 (time 1788394104.65 total 20.03 last 62.14) - open/close 1992 (time 1788394114.69 total 30.07 last 69.04) - open/close 2656 (time 1788394124.69 total 40.07 last 66.39) - open/close 3150 (time 1788394134.73 total 50.11 last 49.22) - open/close 3531 (time 1788394144.73 total 60.11 last 38.10) - open/close 4075 (time 1788394154.77 total 70.15 last 54.18) total: 4500 open/close in 77.26 seconds: 58.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1712 (time 1788394177.74 total 10.00 last 171.12) - /unlink 3412 (time 1788394187.75 total 20.02 last 169.81) total: 4500 /unlink in 28.94 seconds: 155.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 942 (time 1788394211.07 total 10.01 last 94.13) - open/close 1964 (time 1788394221.07 total 20.01 last 102.19) - open/close 2832 (time 1788394231.07 total 30.01 last 86.79) - open/close 3804 (time 1788394241.08 total 40.01 last 97.18) total: 4500 open/close in 47.34 seconds: 95.06 ops/second - /unlink 1664 (time 1788394262.30 total 10.00 last 166.35) - /unlink 3351 (time 1788394272.30 total 20.01 last 168.63) total: 4500 /unlink in 27.11 seconds: 166.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 821 (time 1788394295.44 total 10.01 last 82.06) - open/close 1728 (time 1788394305.45 total 20.01 last 90.63) - open/close 2531 (time 1788394315.45 total 30.02 last 80.27) - open/close 3383 (time 1788394325.48 total 40.04 last 85.00) - open/close 4350 (time 1788394335.48 total 50.04 last 96.67) total: 4500 open/close in 52.66 seconds: 85.45 ops/second - /unlink 1702 (time 1788394352.31 total 10.00 last 170.18) - /unlink 2736 (time 1788394362.32 total 20.01 last 103.31) - /unlink 4383 (time 1788394372.32 total 30.01 last 164.66) total: 4500 /unlink in 30.89 seconds: 145.66 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 94478 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg243-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg243-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg243-server stop mds service on oleg243-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg243-server stop mds service on oleg243-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg243-server unloading modules via unload_modules_local on: 'oleg243-server' oleg243-server: oleg243-server.virtnet: executing unload_modules_local oleg243-server: kmemleak disabled modules unloaded.