== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:27:04 (1788751624) Stopping clients: oleg608-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg608-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg608-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg608-server oleg608-server: oleg608-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing load_modules_local oleg608-server: kmemleak disabled oleg608-server: Loading modules from /home/green/git/lustre-release/lustre oleg608-server: detected 4 online CPUs by sysfs oleg608-server: MODOPTS_LIBCFS= oleg608-server: Force libcfs to create 2 CPU partitions oleg608-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg608-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg608-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg608-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg608-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg608-client.virtnet: -o user_xattr,flock 192.168.206.108@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 790 (time 1788751792.40 total 10.01 last 78.93) - open/close 1551 (time 1788751802.41 total 20.01 last 76.08) - open/close 2293 (time 1788751812.41 total 30.02 last 74.17) - open/close 3036 (time 1788751822.41 total 40.02 last 74.29) - open/close 3796 (time 1788751832.42 total 50.02 last 75.96) - open/close 4496 (time 1788751842.42 total 60.02 last 70.00) total: 4500 open/close in 60.06 seconds: 74.93 ops/second - /unlink 1425 (time 1788751856.51 total 10.00 last 142.49) - /unlink 2910 (time 1788751866.51 total 20.00 last 148.48) - /unlink 4354 (time 1788751876.52 total 30.01 last 144.27) total: 4500 /unlink in 31.18 seconds: 144.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 736 (time 1788751891.54 total 10.01 last 73.53) - open/close 1500 (time 1788751901.54 total 20.01 last 76.36) - open/close 2263 (time 1788751911.54 total 30.02 last 76.29) - open/close 3035 (time 1788751921.55 total 40.02 last 77.16) - open/close 3773 (time 1788751931.56 total 50.03 last 73.74) total: 4500 open/close in 59.68 seconds: 75.40 ops/second - /unlink 1372 (time 1788751955.54 total 10.00 last 137.14) - /unlink 2760 (time 1788751965.54 total 20.01 last 138.77) - /unlink 4087 (time 1788751975.55 total 30.01 last 132.68) total: 4500 /unlink in 32.93 seconds: 136.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 666 (time 1788751993.29 total 10.00 last 66.59) - open/close 1375 (time 1788752003.29 total 20.01 last 70.84) - open/close 2088 (time 1788752013.31 total 30.02 last 71.21) - open/close 2788 (time 1788752023.31 total 40.03 last 69.95) - open/close 3462 (time 1788752033.32 total 50.03 last 67.38) - open/close 4156 (time 1788752043.33 total 60.04 last 69.33) total: 4500 open/close in 65.02 seconds: 69.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1352 (time 1788752062.22 total 10.01 last 135.05) - /unlink 2640 (time 1788752072.22 total 20.01 last 128.79) - /unlink 4059 (time 1788752082.22 total 30.01 last 141.86) total: 4500 /unlink in 33.49 seconds: 134.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 743 (time 1788752100.03 total 10.00 last 74.27) - open/close 1447 (time 1788752110.03 total 20.01 last 70.34) - open/close 2148 (time 1788752120.05 total 30.02 last 70.03) - open/close 2827 (time 1788752130.05 total 40.03 last 67.84) - open/close 3567 (time 1788752140.07 total 50.05 last 73.87) - open/close 4304 (time 1788752150.09 total 60.06 last 73.60) total: 4500 open/close in 62.55 seconds: 71.95 ops/second - /unlink 1513 (time 1788752166.15 total 10.00 last 151.29) - /unlink 3045 (time 1788752176.16 total 20.01 last 153.10) total: 4500 /unlink in 28.41 seconds: 158.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 823 (time 1788752197.50 total 10.02 last 82.11) - open/close 1727 (time 1788752207.50 total 20.02 last 90.40) - open/close 2606 (time 1788752217.53 total 30.05 last 87.68) - open/close 3475 (time 1788752227.55 total 40.07 last 86.69) - open/close 4271 (time 1788752237.56 total 50.08 last 79.55) total: 4500 open/close in 52.80 seconds: 85.23 ops/second - /unlink 1699 (time 1788752253.43 total 10.00 last 169.83) - /unlink 3402 (time 1788752263.44 total 20.01 last 170.21) total: 4500 /unlink in 26.95 seconds: 166.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 840 (time 1788752283.65 total 10.00 last 83.98) - open/close 1735 (time 1788752293.65 total 20.01 last 89.45) - open/close 2582 (time 1788752303.66 total 30.02 last 84.61) - open/close 3417 (time 1788752313.68 total 40.03 last 83.40) - open/close 4277 (time 1788752323.68 total 50.03 last 85.98) total: 4500 open/close in 52.42 seconds: 85.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1551 (time 1788752338.85 total 10.00 last 155.07) - /unlink 3137 (time 1788752348.86 total 20.01 last 158.49) total: 4500 /unlink in 29.39 seconds: 153.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 823 (time 1788752371.81 total 10.01 last 82.24) - open/close 1610 (time 1788752381.82 total 20.02 last 78.63) - open/close 2418 (time 1788752391.82 total 30.02 last 80.76) - open/close 3211 (time 1788752401.83 total 40.03 last 79.23) - open/close 4129 (time 1788752411.84 total 50.04 last 91.74) total: 4500 open/close in 53.64 seconds: 83.90 ops/second - /unlink 1811 (time 1788752428.17 total 10.00 last 181.08) - /unlink 3529 (time 1788752438.17 total 20.00 last 171.79) total: 4500 /unlink in 24.18 seconds: 186.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1213 (time 1788752454.61 total 10.00 last 121.27) - open/close 2279 (time 1788752464.61 total 20.01 last 106.52) - open/close 3290 (time 1788752474.62 total 30.01 last 101.06) - open/close 4434 (time 1788752484.63 total 40.03 last 114.25) total: 4500 open/close in 40.74 seconds: 110.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2057 (time 1788752497.69 total 10.00 last 205.69) - /unlink 4152 (time 1788752507.69 total 20.00 last 209.43) total: 4500 /unlink in 21.46 seconds: 209.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1005 (time 1788752522.06 total 10.00 last 100.48) - open/close 2108 (time 1788752532.07 total 20.01 last 110.26) - open/close 3399 (time 1788752542.07 total 30.01 last 129.08) total: 4500 open/close in 39.45 seconds: 114.06 ops/second - /unlink 2330 (time 1788752563.61 total 10.00 last 232.98) total: 4500 /unlink in 18.45 seconds: 243.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1161 (time 1788752584.52 total 10.01 last 116.02) - open/close 2232 (time 1788752594.53 total 20.01 last 107.05) - open/close 3350 (time 1788752604.53 total 30.02 last 111.75) - open/close 4346 (time 1788752614.54 total 40.02 last 99.53) total: 4500 open/close in 41.36 seconds: 108.81 ops/second - /unlink 2139 (time 1788752628.06 total 10.00 last 213.85) - /unlink 4314 (time 1788752638.06 total 20.01 last 217.41) total: 4500 /unlink in 20.97 seconds: 214.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1189 (time 1788752651.54 total 10.00 last 118.86) - open/close 2299 (time 1788752661.54 total 20.01 last 110.97) - open/close 3500 (time 1788752671.55 total 30.01 last 120.07) total: 4500 open/close in 38.03 seconds: 118.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2142 (time 1788752691.92 total 10.00 last 214.11) - /unlink 4315 (time 1788752701.93 total 20.01 last 217.24) total: 4500 /unlink in 20.84 seconds: 215.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1167 (time 1788752714.96 total 10.00 last 116.65) - open/close 2272 (time 1788752724.97 total 20.01 last 110.42) - open/close 3341 (time 1788752734.97 total 30.01 last 106.89) - open/close 4492 (time 1788752744.98 total 40.02 last 114.96) total: 4500 open/close in 40.07 seconds: 112.30 ops/second - /unlink 2054 (time 1788752757.24 total 10.01 last 205.22) - /unlink 4206 (time 1788752767.24 total 20.01 last 215.13) total: 4500 /unlink in 21.42 seconds: 210.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1477 (time 1788752780.78 total 10.01 last 147.61) - open/close 2742 (time 1788752790.79 total 20.01 last 126.49) - open/close 3719 (time 1788752800.79 total 30.01 last 97.69) total: 4500 open/close in 36.60 seconds: 122.97 ops/second - /unlink 2566 (time 1788752819.30 total 10.00 last 256.53) total: 4500 /unlink in 18.98 seconds: 237.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1506 (time 1788752840.38 total 10.00 last 150.57) - open/close 2882 (time 1788752850.39 total 20.00 last 137.57) - open/close 4163 (time 1788752860.39 total 30.01 last 128.03) total: 4500 open/close in 32.78 seconds: 137.27 ops/second - /unlink 2371 (time 1788752875.33 total 10.00 last 236.99) total: 4500 /unlink in 19.26 seconds: 233.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1233 (time 1788752896.68 total 10.00 last 123.26) - open/close 2573 (time 1788752906.68 total 20.01 last 133.96) - open/close 3731 (time 1788752916.68 total 30.01 last 115.78) total: 4500 open/close in 37.39 seconds: 120.35 ops/second - /unlink 2164 (time 1788752936.67 total 10.00 last 216.39) - /unlink 4186 (time 1788752946.68 total 20.01 last 202.11) total: 4500 /unlink in 21.68 seconds: 207.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1046 (time 1788752961.12 total 10.00 last 104.55) - open/close 2204 (time 1788752971.13 total 20.01 last 115.70) - open/close 3560 (time 1788752981.13 total 30.02 last 135.57) total: 4500 open/close in 37.31 seconds: 120.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2255 (time 1788753000.67 total 10.00 last 225.45) - /unlink 4425 (time 1788753010.68 total 20.00 last 216.95) total: 4500 /unlink in 20.47 seconds: 219.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1068 (time 1788753023.68 total 10.00 last 106.77) - open/close 2073 (time 1788753033.68 total 20.01 last 100.47) - open/close 3116 (time 1788753043.68 total 30.01 last 104.30) - open/close 4296 (time 1788753053.68 total 40.01 last 117.94) total: 4500 open/close in 41.91 seconds: 107.38 ops/second - /unlink 2436 (time 1788753068.00 total 10.00 last 243.55) total: 4500 /unlink in 18.45 seconds: 243.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1330 (time 1788753088.70 total 10.00 last 132.99) - open/close 2764 (time 1788753098.70 total 20.00 last 143.36) - open/close 4217 (time 1788753108.70 total 30.01 last 145.27) total: 4500 open/close in 31.95 seconds: 140.86 ops/second - /unlink 4427 (time 1788753122.55 total 10.00 last 442.64) total: 4500 /unlink in 10.23 seconds: 439.75 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 94967 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg608-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server unloading modules via unload_modules_local on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing unload_modules_local oleg608-server: kmemleak disabled modules unloaded.