== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:16:13 (1789431373) Stopping clients: oleg212-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg212-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg212-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg212-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg212-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg212-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg212-server oleg212-server: oleg212-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg212-server' oleg212-server: oleg212-server.virtnet: executing load_modules_local oleg212-server: kmemleak disabled oleg212-server: Loading modules from /home/green/git/lustre-release/lustre oleg212-server: detected 4 online CPUs by sysfs oleg212-server: MODOPTS_LIBCFS= oleg212-server: Force libcfs to create 2 CPU partitions oleg212-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg212-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg212-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg212-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg212-server: oleg212-server.virtnet: executing set_default_debug -1 all pdsh@oleg212-client: oleg212-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg212-client.virtnet: -o user_xattr,flock 192.168.202.112@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 727 (time 1789431565.72 total 10.01 last 72.66) - open/close 1414 (time 1789431575.73 total 20.01 last 68.66) - open/close 2240 (time 1789431585.73 total 30.01 last 82.59) - open/close 2992 (time 1789431595.74 total 40.02 last 75.14) - open/close 3780 (time 1789431605.74 total 50.03 last 78.76) total: 4500 open/close in 58.04 seconds: 77.54 ops/second - /unlink 1635 (time 1789431627.89 total 10.00 last 163.45) - /unlink 3196 (time 1789431637.89 total 20.01 last 156.01) total: 4500 /unlink in 29.17 seconds: 154.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 873 (time 1789431661.77 total 10.01 last 87.25) - open/close 1759 (time 1789431671.77 total 20.01 last 88.60) - open/close 2673 (time 1789431681.77 total 30.01 last 91.36) - open/close 3483 (time 1789431691.79 total 40.03 last 80.86) - open/close 4254 (time 1789431701.79 total 50.03 last 77.09) total: 4500 open/close in 53.32 seconds: 84.40 ops/second - /unlink 1765 (time 1789431719.11 total 10.00 last 176.47) - /unlink 3680 (time 1789431729.11 total 20.00 last 191.49) total: 4500 /unlink in 24.77 seconds: 181.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 927 (time 1789431747.88 total 10.00 last 92.67) - open/close 1849 (time 1789431757.89 total 20.01 last 92.18) - open/close 2808 (time 1789431767.89 total 30.01 last 95.83) - open/close 3856 (time 1789431777.90 total 40.01 last 104.78) total: 4500 open/close in 45.87 seconds: 98.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1511 (time 1789431797.62 total 10.01 last 150.91) - /unlink 3366 (time 1789431807.63 total 20.02 last 185.42) total: 4500 /unlink in 25.79 seconds: 174.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 878 (time 1789431826.56 total 10.00 last 87.78) - open/close 1806 (time 1789431836.56 total 20.00 last 92.79) - open/close 2710 (time 1789431846.57 total 30.01 last 90.38) - open/close 3588 (time 1789431856.57 total 40.01 last 87.80) - open/close 4496 (time 1789431866.57 total 50.01 last 90.73) total: 4500 open/close in 50.06 seconds: 89.90 ops/second - /unlink 1604 (time 1789431881.11 total 10.00 last 160.37) - /unlink 3196 (time 1789431891.13 total 20.02 last 158.85) total: 4500 /unlink in 27.52 seconds: 163.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 947 (time 1789431912.70 total 10.01 last 94.63) - open/close 1892 (time 1789431922.71 total 20.02 last 94.41) - open/close 2907 (time 1789431932.71 total 30.02 last 101.45) - open/close 3871 (time 1789431942.71 total 40.02 last 96.38) total: 4500 open/close in 46.13 seconds: 97.54 ops/second - /unlink 1681 (time 1789431962.76 total 10.01 last 167.99) - /unlink 3414 (time 1789431972.77 total 20.01 last 173.23) total: 4500 /unlink in 26.47 seconds: 169.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 888 (time 1789431993.38 total 10.00 last 88.79) - open/close 1852 (time 1789432003.38 total 20.00 last 96.37) - open/close 2788 (time 1789432013.39 total 30.01 last 93.58) - open/close 3725 (time 1789432023.40 total 40.02 last 93.58) - open/close 4447 (time 1789432033.42 total 50.04 last 72.03) total: 4500 open/close in 51.11 seconds: 88.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1628 (time 1789432048.75 total 10.00 last 162.74) - /unlink 3379 (time 1789432058.77 total 20.03 last 174.70) total: 4500 /unlink in 27.25 seconds: 165.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 758 (time 1789432082.92 total 10.01 last 75.73) - open/close 1561 (time 1789432092.92 total 20.02 last 80.24) - open/close 2378 (time 1789432102.94 total 30.03 last 81.60) - open/close 3320 (time 1789432112.94 total 40.03 last 94.15) - open/close 4241 (time 1789432122.94 total 50.03 last 92.10) total: 4500 open/close in 52.98 seconds: 84.94 ops/second - /unlink 1729 (time 1789432139.85 total 10.01 last 172.81) - /unlink 3202 (time 1789432149.85 total 20.01 last 147.23) total: 4500 /unlink in 27.53 seconds: 163.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 710 (time 1789432171.31 total 10.02 last 70.87) - open/close 1636 (time 1789432181.31 total 20.02 last 92.59) - open/close 2541 (time 1789432191.31 total 30.02 last 90.50) - open/close 3421 (time 1789432201.35 total 40.06 last 87.64) - open/close 4276 (time 1789432211.35 total 50.06 last 85.50) total: 4500 open/close in 52.95 seconds: 84.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1885 (time 1789432227.95 total 10.00 last 188.47) - /unlink 3431 (time 1789432237.95 total 20.00 last 154.59) total: 4500 /unlink in 27.57 seconds: 163.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 834 (time 1789432260.58 total 10.00 last 83.37) - open/close 1686 (time 1789432270.58 total 20.01 last 85.15) - open/close 2482 (time 1789432280.59 total 30.02 last 79.51) - open/close 3396 (time 1789432290.60 total 40.03 last 91.30) - open/close 4356 (time 1789432300.61 total 50.04 last 95.95) total: 4500 open/close in 51.96 seconds: 86.60 ops/second - /unlink 1394 (time 1789432316.71 total 10.00 last 139.34) - /unlink 2873 (time 1789432326.71 total 20.01 last 147.84) total: 4500 /unlink in 29.40 seconds: 153.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 771 (time 1789432350.47 total 10.02 last 76.92) - open/close 1692 (time 1789432360.47 total 20.02 last 92.08) - open/close 2661 (time 1789432370.47 total 30.03 last 96.87) - open/close 3592 (time 1789432380.49 total 40.05 last 92.91) total: 4500 open/close in 49.53 seconds: 90.86 ops/second - /unlink 1443 (time 1789432403.68 total 10.01 last 144.09) - /unlink 2751 (time 1789432413.69 total 20.02 last 130.75) - /unlink 4203 (time 1789432423.69 total 30.02 last 145.15) total: 4500 /unlink in 31.90 seconds: 141.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 638 (time 1789432440.79 total 10.01 last 63.74) - open/close 1512 (time 1789432450.80 total 20.01 last 87.36) - open/close 2355 (time 1789432460.80 total 30.02 last 84.28) - open/close 3313 (time 1789432470.80 total 40.02 last 95.80) - open/close 4240 (time 1789432480.81 total 50.03 last 92.63) total: 4500 open/close in 52.75 seconds: 85.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1906 (time 1789432497.38 total 10.00 last 190.56) - /unlink 3549 (time 1789432507.39 total 20.01 last 164.12) total: 4500 /unlink in 25.82 seconds: 174.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 874 (time 1789432526.92 total 10.00 last 87.39) - open/close 1662 (time 1789432536.93 total 20.01 last 78.76) - open/close 2446 (time 1789432546.93 total 30.01 last 78.37) - open/close 3363 (time 1789432556.93 total 40.01 last 91.70) - open/close 4210 (time 1789432566.94 total 50.02 last 84.61) total: 4500 open/close in 53.26 seconds: 84.49 ops/second - /unlink 1780 (time 1789432584.39 total 10.00 last 177.93) - /unlink 3262 (time 1789432594.39 total 20.01 last 148.16) total: 4500 /unlink in 27.00 seconds: 166.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 916 (time 1789432615.54 total 10.01 last 91.53) - open/close 1902 (time 1789432625.57 total 20.04 last 98.32) - open/close 2733 (time 1789432635.58 total 30.05 last 83.00) - open/close 3657 (time 1789432645.58 total 40.05 last 92.39) - open/close 4454 (time 1789432655.59 total 50.06 last 79.65) total: 4500 open/close in 50.44 seconds: 89.22 ops/second - /unlink 1664 (time 1789432670.09 total 10.01 last 166.27) - /unlink 3310 (time 1789432680.09 total 20.01 last 164.50) total: 4500 /unlink in 28.01 seconds: 160.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 990 (time 1789432702.06 total 10.01 last 98.95) - open/close 1967 (time 1789432712.07 total 20.01 last 97.62) - open/close 2896 (time 1789432722.08 total 30.02 last 92.81) - open/close 3921 (time 1789432732.09 total 40.04 last 102.34) total: 4500 open/close in 46.79 seconds: 96.18 ops/second - /unlink 1790 (time 1789432753.20 total 10.00 last 179.00) - /unlink 3503 (time 1789432763.20 total 20.00 last 171.28) total: 4500 /unlink in 26.05 seconds: 172.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 927 (time 1789432783.41 total 10.01 last 92.65) - open/close 1813 (time 1789432793.42 total 20.01 last 88.54) - open/close 2660 (time 1789432803.44 total 30.03 last 84.53) - open/close 3585 (time 1789432813.44 total 40.04 last 92.47) - open/close 4472 (time 1789432823.45 total 50.05 last 88.61) total: 4500 open/close in 50.37 seconds: 89.35 ops/second - /unlink 1534 (time 1789432838.10 total 10.01 last 153.32) - /unlink 3549 (time 1789432848.10 total 20.01 last 201.46) total: 4500 /unlink in 26.39 seconds: 170.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 826 (time 1789432869.65 total 10.00 last 82.57) - open/close 1590 (time 1789432879.65 total 20.01 last 76.39) - open/close 2526 (time 1789432889.65 total 30.01 last 93.58) - open/close 3469 (time 1789432899.66 total 40.01 last 94.24) - open/close 4432 (time 1789432909.66 total 50.01 last 96.29) total: 4500 open/close in 50.78 seconds: 88.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1839 (time 1789432924.93 total 10.00 last 183.89) - /unlink 3569 (time 1789432934.93 total 20.00 last 172.96) total: 4500 /unlink in 26.27 seconds: 171.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 852 (time 1789432954.63 total 10.01 last 85.13) - open/close 1749 (time 1789432964.64 total 20.02 last 89.56) - open/close 2661 (time 1789432974.66 total 30.04 last 91.06) - open/close 3412 (time 1789432984.69 total 40.07 last 74.89) - open/close 4217 (time 1789432994.70 total 50.08 last 80.41) total: 4500 open/close in 52.88 seconds: 85.11 ops/second - /unlink 1826 (time 1789433011.61 total 10.00 last 182.58) - /unlink 3513 (time 1789433021.62 total 20.00 last 168.67) total: 4500 /unlink in 25.22 seconds: 178.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 989 (time 1789433040.43 total 10.02 last 98.74) - open/close 1939 (time 1789433050.43 total 20.02 last 94.94) - open/close 2960 (time 1789433060.44 total 30.03 last 102.01) - open/close 4017 (time 1789433070.45 total 40.04 last 105.60) total: 4500 open/close in 45.26 seconds: 99.42 ops/second - /unlink 1615 (time 1789433089.90 total 10.00 last 161.50) - /unlink 3072 (time 1789433099.90 total 20.00 last 145.67) total: 4500 /unlink in 28.77 seconds: 156.40 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95222 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg212-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg212-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg212-server stop mds service on oleg212-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg212-server stop mds service on oleg212-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg212-server unloading modules via unload_modules_local on: 'oleg212-server' oleg212-server: oleg212-server.virtnet: executing unload_modules_local oleg212-server: kmemleak disabled modules unloaded.