== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:31:03 (1788654663) Stopping clients: oleg616-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg616-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg616-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg616-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg616-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg616-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg616-server oleg616-server: oleg616-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg616-server' oleg616-server: oleg616-server.virtnet: executing load_modules_local oleg616-server: kmemleak disabled oleg616-server: Loading modules from /home/green/git/lustre-release/lustre oleg616-server: detected 4 online CPUs by sysfs oleg616-server: MODOPTS_LIBCFS= oleg616-server: Force libcfs to create 2 CPU partitions oleg616-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg616-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg616-server: oleg616-server.virtnet: executing set_default_debug -1 all pdsh@oleg616-client: oleg616-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg616-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg616-server: oleg616-server.virtnet: executing set_default_debug -1 all pdsh@oleg616-client: oleg616-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg616-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg616-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg616-server: oleg616-server.virtnet: executing set_default_debug -1 all pdsh@oleg616-client: oleg616-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg616-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg616-client.virtnet: -o user_xattr,flock 192.168.206.116@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 918 (time 1788654842.99 total 10.01 last 91.72) - open/close 1767 (time 1788654852.99 total 20.01 last 84.89) - open/close 2655 (time 1788654862.99 total 30.01 last 88.75) - open/close 3558 (time 1788654873.00 total 40.02 last 90.25) - open/close 4375 (time 1788654883.00 total 50.02 last 81.69) total: 4500 open/close in 51.55 seconds: 87.29 ops/second - /unlink 1751 (time 1788654898.25 total 10.01 last 175.00) - /unlink 3308 (time 1788654908.26 total 20.02 last 155.54) total: 4500 /unlink in 27.35 seconds: 164.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 938 (time 1788654929.83 total 10.01 last 93.75) - open/close 1844 (time 1788654939.84 total 20.02 last 90.48) - open/close 2729 (time 1788654949.84 total 30.02 last 88.50) - open/close 3710 (time 1788654959.85 total 40.02 last 98.05) total: 4500 open/close in 48.99 seconds: 91.85 ops/second - /unlink 1638 (time 1788654983.13 total 10.00 last 163.76) - /unlink 3242 (time 1788654993.13 total 20.00 last 160.38) total: 4500 /unlink in 27.45 seconds: 163.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 942 (time 1788655015.03 total 10.00 last 94.16) - open/close 1841 (time 1788655025.03 total 20.01 last 89.86) - open/close 2743 (time 1788655035.04 total 30.01 last 90.15) - open/close 3608 (time 1788655045.05 total 40.02 last 86.43) - open/close 4477 (time 1788655055.05 total 50.03 last 86.85) total: 4500 open/close in 50.22 seconds: 89.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1606 (time 1788655069.47 total 10.00 last 160.54) - /unlink 3155 (time 1788655079.47 total 20.01 last 154.86) total: 4500 /unlink in 28.33 seconds: 158.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 957 (time 1788655102.54 total 10.02 last 95.54) - open/close 1842 (time 1788655112.55 total 20.02 last 88.45) - open/close 2743 (time 1788655122.55 total 30.03 last 90.06) - open/close 3534 (time 1788655132.56 total 40.03 last 79.09) - open/close 4416 (time 1788655142.57 total 50.04 last 88.11) total: 4500 open/close in 51.10 seconds: 88.06 ops/second - /unlink 1588 (time 1788655158.09 total 10.00 last 158.80) - /unlink 3116 (time 1788655168.09 total 20.00 last 152.77) total: 4500 /unlink in 28.42 seconds: 158.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 803 (time 1788655190.84 total 10.01 last 80.24) - open/close 1636 (time 1788655200.84 total 20.01 last 83.27) - open/close 2419 (time 1788655210.86 total 30.03 last 78.17) - open/close 3203 (time 1788655220.86 total 40.03 last 78.39) - open/close 4090 (time 1788655230.87 total 50.04 last 88.60) total: 4500 open/close in 54.31 seconds: 82.85 ops/second - /unlink 1605 (time 1788655249.03 total 10.00 last 160.45) - /unlink 3298 (time 1788655259.04 total 20.01 last 169.24) total: 4500 /unlink in 28.23 seconds: 159.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 881 (time 1788655281.60 total 10.01 last 87.98) - open/close 1729 (time 1788655291.60 total 20.01 last 84.79) - open/close 2558 (time 1788655301.60 total 30.02 last 82.89) - open/close 3482 (time 1788655311.61 total 40.03 last 92.28) - open/close 4360 (time 1788655321.61 total 50.03 last 87.77) total: 4500 open/close in 51.82 seconds: 86.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1652 (time 1788655337.40 total 10.00 last 165.20) - /unlink 3337 (time 1788655347.40 total 20.01 last 168.39) total: 4500 /unlink in 27.11 seconds: 166.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 881 (time 1788655368.36 total 10.01 last 88.01) - open/close 1784 (time 1788655378.36 total 20.02 last 90.25) - open/close 2701 (time 1788655388.37 total 30.02 last 91.68) - open/close 3598 (time 1788655398.37 total 40.02 last 89.69) - open/close 4433 (time 1788655408.38 total 50.03 last 83.38) total: 4500 open/close in 50.83 seconds: 88.53 ops/second - /unlink 1616 (time 1788655423.32 total 10.00 last 161.55) - /unlink 3201 (time 1788655433.33 total 20.01 last 158.42) total: 4500 /unlink in 27.39 seconds: 164.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 871 (time 1788655455.19 total 10.00 last 87.07) - open/close 1777 (time 1788655465.19 total 20.01 last 90.55) - open/close 2763 (time 1788655475.19 total 30.01 last 98.59) - open/close 3601 (time 1788655485.20 total 40.01 last 83.79) total: 4500 open/close in 49.69 seconds: 90.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1716 (time 1788655508.89 total 10.00 last 171.60) - /unlink 3372 (time 1788655518.89 total 20.00 last 165.55) total: 4500 /unlink in 26.67 seconds: 168.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 815 (time 1788655540.21 total 10.01 last 81.46) - open/close 1758 (time 1788655550.21 total 20.02 last 94.21) - open/close 2706 (time 1788655560.22 total 30.02 last 94.79) - open/close 3630 (time 1788655570.22 total 40.02 last 92.35) total: 4500 open/close in 49.05 seconds: 91.75 ops/second - /unlink 1772 (time 1788655592.51 total 10.01 last 177.11) - /unlink 3436 (time 1788655602.51 total 20.01 last 166.37) total: 4500 /unlink in 26.25 seconds: 171.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 862 (time 1788655622.68 total 10.02 last 86.00) - open/close 1803 (time 1788655632.68 total 20.03 last 94.08) - open/close 2717 (time 1788655642.68 total 30.03 last 91.37) - open/close 3632 (time 1788655652.69 total 40.03 last 91.45) total: 4500 open/close in 49.13 seconds: 91.60 ops/second - /unlink 1709 (time 1788655675.21 total 10.00 last 170.86) - /unlink 3279 (time 1788655685.21 total 20.01 last 156.93) total: 4500 /unlink in 27.70 seconds: 162.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 900 (time 1788655707.37 total 10.01 last 89.95) - open/close 1825 (time 1788655717.38 total 20.01 last 92.42) - open/close 2741 (time 1788655727.39 total 30.02 last 91.55) - open/close 3659 (time 1788655737.39 total 40.02 last 91.79) total: 4500 open/close in 49.67 seconds: 90.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1658 (time 1788655760.17 total 10.00 last 165.74) - /unlink 3115 (time 1788655770.18 total 20.01 last 145.56) total: 4500 /unlink in 29.40 seconds: 153.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 859 (time 1788655794.27 total 10.01 last 85.86) - open/close 1821 (time 1788655804.28 total 20.01 last 96.13) - open/close 2769 (time 1788655814.29 total 30.02 last 94.70) - open/close 3669 (time 1788655824.29 total 40.02 last 89.98) total: 4500 open/close in 49.73 seconds: 90.49 ops/second - /unlink 1694 (time 1788655848.13 total 10.00 last 169.36) - /unlink 3379 (time 1788655858.13 total 20.01 last 168.41) total: 4500 /unlink in 25.77 seconds: 174.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 659 (time 1788655878.11 total 10.01 last 65.85) - open/close 1184 (time 1788655888.11 total 20.02 last 52.46) - open/close 1724 (time 1788655898.13 total 30.03 last 53.92) - open/close 2364 (time 1788655908.13 total 40.04 last 63.97) - open/close 3173 (time 1788655918.14 total 50.04 last 80.86) - open/close 3989 (time 1788655928.14 total 60.04 last 81.58) total: 4500 open/close in 65.48 seconds: 68.72 ops/second - /unlink 1631 (time 1788655947.27 total 10.00 last 163.03) - /unlink 3145 (time 1788655957.27 total 20.01 last 151.37) total: 4500 /unlink in 27.88 seconds: 161.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 801 (time 1788655979.43 total 10.00 last 80.07) - open/close 1643 (time 1788655989.43 total 20.01 last 84.17) - open/close 2609 (time 1788655999.44 total 30.01 last 96.54) - open/close 3445 (time 1788656009.44 total 40.01 last 83.59) - open/close 4353 (time 1788656019.44 total 50.02 last 90.79) total: 4500 open/close in 51.68 seconds: 87.07 ops/second - /unlink 1724 (time 1788656034.85 total 10.00 last 172.38) - /unlink 3351 (time 1788656044.85 total 20.00 last 162.65) total: 4500 /unlink in 27.13 seconds: 165.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 894 (time 1788656065.68 total 10.01 last 89.31) - open/close 1820 (time 1788656075.68 total 20.01 last 92.57) - open/close 2722 (time 1788656085.69 total 30.02 last 90.11) - open/close 3590 (time 1788656095.71 total 40.04 last 86.68) - open/close 4496 (time 1788656105.71 total 50.04 last 90.59) total: 4500 open/close in 50.08 seconds: 89.86 ops/second - /unlink 1708 (time 1788656120.10 total 10.00 last 170.75) - /unlink 3406 (time 1788656130.10 total 20.00 last 169.77) total: 4500 /unlink in 26.75 seconds: 168.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 883 (time 1788656150.70 total 10.00 last 88.26) - open/close 1642 (time 1788656160.71 total 20.01 last 75.85) - open/close 2411 (time 1788656170.72 total 30.02 last 76.83) - open/close 3296 (time 1788656180.73 total 40.03 last 88.44) - open/close 4139 (time 1788656190.73 total 50.03 last 84.26) total: 4500 open/close in 54.86 seconds: 82.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1609 (time 1788656209.84 total 10.01 last 160.75) - /unlink 3215 (time 1788656219.84 total 20.01 last 160.57) total: 4500 /unlink in 28.17 seconds: 159.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 853 (time 1788656242.74 total 10.00 last 85.27) - open/close 1738 (time 1788656252.75 total 20.01 last 88.45) - open/close 2607 (time 1788656262.75 total 30.01 last 86.89) - open/close 3527 (time 1788656272.75 total 40.01 last 92.00) - open/close 4379 (time 1788656282.76 total 50.02 last 85.15) total: 4500 open/close in 51.26 seconds: 87.79 ops/second - /unlink 1432 (time 1788656297.84 total 10.01 last 143.10) - /unlink 2723 (time 1788656307.85 total 20.01 last 129.02) - /unlink 4249 (time 1788656317.85 total 30.02 last 152.53) total: 4500 /unlink in 31.94 seconds: 140.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 797 (time 1788656333.97 total 10.00 last 79.67) - open/close 1492 (time 1788656343.98 total 20.01 last 69.43) - open/close 2288 (time 1788656353.99 total 30.02 last 79.51) - open/close 2883 (time 1788656363.99 total 40.03 last 59.47) - open/close 3579 (time 1788656374.00 total 50.03 last 69.56) - open/close 4352 (time 1788656384.01 total 60.04 last 77.24) total: 4500 open/close in 62.06 seconds: 72.51 ops/second - /unlink 1448 (time 1788656400.70 total 10.00 last 144.77) - /unlink 2965 (time 1788656410.71 total 20.01 last 151.64) total: 4500 /unlink in 29.68 seconds: 151.62 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95017 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg616-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg616-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg616-server stop mds service on oleg616-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg616-server stop mds service on oleg616-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg616-server unloading modules via unload_modules_local on: 'oleg616-server' oleg616-server: oleg616-server.virtnet: executing unload_modules_local oleg616-server: kmemleak disabled modules unloaded.