== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:38:31 (1783139911) Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg124-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg124-server oleg124-server: oleg124-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg124-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 822 (time 1783140127.17 total 10.01 last 82.11) - open/close 1400 (time 1783140137.17 total 20.01 last 57.79) - open/close 2109 (time 1783140147.19 total 30.03 last 70.81) - open/close 2880 (time 1783140157.19 total 40.03 last 77.10) - open/close 3559 (time 1783140167.19 total 50.03 last 67.87) - open/close 4255 (time 1783140177.21 total 60.05 last 69.48) total: 4500 open/close in 63.49 seconds: 70.88 ops/second - /unlink 1455 (time 1783140195.31 total 10.00 last 145.46) - /unlink 2733 (time 1783140205.32 total 20.01 last 127.76) - /unlink 4056 (time 1783140215.32 total 30.01 last 132.28) total: 4500 /unlink in 32.78 seconds: 137.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 734 (time 1783140232.79 total 10.00 last 73.39) - open/close 1477 (time 1783140242.79 total 20.00 last 74.28) - open/close 2282 (time 1783140252.79 total 30.00 last 80.49) - open/close 3012 (time 1783140262.81 total 40.02 last 72.90) - open/close 3670 (time 1783140272.81 total 50.03 last 65.75) - open/close 4324 (time 1783140282.83 total 60.04 last 65.33) total: 4500 open/close in 62.83 seconds: 71.62 ops/second - /unlink 1161 (time 1783140300.40 total 10.00 last 116.06) - /unlink 2364 (time 1783140310.40 total 20.00 last 120.28) - /unlink 3285 (time 1783140320.40 total 30.01 last 92.06) total: 4500 /unlink in 39.84 seconds: 112.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 734 (time 1783140345.01 total 10.01 last 73.32) - open/close 1418 (time 1783140355.03 total 20.03 last 68.29) - open/close 2150 (time 1783140365.03 total 30.03 last 73.19) - open/close 2952 (time 1783140375.03 total 40.03 last 80.19) - open/close 3742 (time 1783140385.03 total 50.04 last 78.95) total: 4500 open/close in 59.57 seconds: 75.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1425 (time 1783140408.97 total 10.00 last 142.49) - /unlink 2869 (time 1783140418.97 total 20.00 last 144.37) - /unlink 4170 (time 1783140428.97 total 30.00 last 130.10) total: 4500 /unlink in 32.41 seconds: 138.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 770 (time 1783140445.93 total 10.02 last 76.87) - open/close 1411 (time 1783140455.93 total 20.02 last 64.09) - open/close 2136 (time 1783140465.93 total 30.02 last 72.48) - open/close 2864 (time 1783140475.94 total 40.03 last 72.74) - open/close 3686 (time 1783140485.95 total 50.04 last 82.10) - open/close 4478 (time 1783140495.95 total 60.04 last 79.19) total: 4500 open/close in 60.29 seconds: 74.64 ops/second - /unlink 1477 (time 1783140510.29 total 10.00 last 147.69) - /unlink 3114 (time 1783140520.31 total 20.02 last 163.43) total: 4500 /unlink in 28.85 seconds: 155.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 877 (time 1783140543.25 total 10.00 last 87.66) - open/close 1721 (time 1783140553.26 total 20.01 last 84.39) - open/close 2620 (time 1783140563.26 total 30.01 last 89.89) - open/close 3399 (time 1783140573.27 total 40.02 last 77.84) - open/close 4249 (time 1783140583.27 total 50.02 last 84.96) total: 4500 open/close in 53.24 seconds: 84.52 ops/second - /unlink 1705 (time 1783140600.28 total 10.00 last 170.50) - /unlink 3170 (time 1783140610.28 total 20.01 last 146.41) total: 4500 /unlink in 29.09 seconds: 154.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 757 (time 1783140633.43 total 10.00 last 75.68) - open/close 1601 (time 1783140643.44 total 20.01 last 84.33) - open/close 2508 (time 1783140653.44 total 30.02 last 90.63) - open/close 3344 (time 1783140663.45 total 40.02 last 83.56) - open/close 4132 (time 1783140673.46 total 50.03 last 78.74) total: 4500 open/close in 54.70 seconds: 82.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1389 (time 1783140691.69 total 10.02 last 138.69) - /unlink 2940 (time 1783140701.69 total 20.02 last 155.09) total: 4500 /unlink in 29.84 seconds: 150.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 817 (time 1783140725.11 total 10.00 last 81.67) - open/close 1703 (time 1783140735.12 total 20.01 last 88.52) - open/close 2709 (time 1783140745.13 total 30.02 last 100.52) - open/close 3654 (time 1783140755.14 total 40.03 last 94.40) total: 4500 open/close in 49.38 seconds: 91.13 ops/second - /unlink 1538 (time 1783140778.19 total 10.00 last 153.74) - /unlink 3137 (time 1783140788.19 total 20.00 last 159.89) total: 4500 /unlink in 29.54 seconds: 152.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 973 (time 1783140811.82 total 10.01 last 97.21) - open/close 1917 (time 1783140821.82 total 20.01 last 94.37) - open/close 2755 (time 1783140831.82 total 30.01 last 83.79) - open/close 3604 (time 1783140841.83 total 40.02 last 84.82) - open/close 4408 (time 1783140851.85 total 50.04 last 80.25) total: 4500 open/close in 51.04 seconds: 88.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1692 (time 1783140865.87 total 10.00 last 169.14) - /unlink 3664 (time 1783140875.87 total 20.01 last 197.16) total: 4500 /unlink in 24.93 seconds: 180.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1038 (time 1783140893.62 total 10.01 last 103.74) - open/close 2083 (time 1783140903.63 total 20.02 last 104.39) - open/close 2926 (time 1783140913.64 total 30.02 last 84.26) - open/close 3827 (time 1783140923.64 total 40.02 last 90.08) total: 4500 open/close in 47.52 seconds: 94.70 ops/second - /unlink 1721 (time 1783140944.03 total 10.00 last 172.05) - /unlink 3266 (time 1783140954.03 total 20.00 last 154.48) total: 4500 /unlink in 28.60 seconds: 157.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 915 (time 1783140977.15 total 10.02 last 91.36) - open/close 1909 (time 1783140987.16 total 20.03 last 99.30) - open/close 2886 (time 1783140997.16 total 30.03 last 97.68) - open/close 3838 (time 1783141007.16 total 40.03 last 95.20) total: 4500 open/close in 46.58 seconds: 96.60 ops/second - /unlink 1935 (time 1783141026.58 total 10.00 last 193.49) - /unlink 3676 (time 1783141036.59 total 20.01 last 173.98) total: 4500 /unlink in 24.47 seconds: 183.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1003 (time 1783141054.37 total 10.00 last 100.26) - open/close 1952 (time 1783141064.37 total 20.00 last 94.89) - open/close 2924 (time 1783141074.37 total 30.01 last 97.18) - open/close 3838 (time 1783141084.38 total 40.02 last 91.29) total: 4500 open/close in 46.95 seconds: 95.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1961 (time 1783141104.00 total 10.00 last 196.07) - /unlink 3813 (time 1783141114.01 total 20.01 last 185.10) total: 4500 /unlink in 23.28 seconds: 193.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1242 (time 1783141129.65 total 10.00 last 124.18) - open/close 2758 (time 1783141139.65 total 20.00 last 151.55) - open/close 3920 (time 1783141149.66 total 30.01 last 116.09) total: 4500 open/close in 36.27 seconds: 124.06 ops/second - /unlink 2430 (time 1783141168.95 total 10.00 last 242.96) - /unlink 4396 (time 1783141178.96 total 20.01 last 196.46) total: 4500 /unlink in 20.55 seconds: 218.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 881 (time 1783141192.74 total 10.00 last 88.06) - open/close 1794 (time 1783141202.75 total 20.01 last 91.23) - open/close 2716 (time 1783141212.76 total 30.02 last 92.16) - open/close 3724 (time 1783141222.76 total 40.02 last 100.79) total: 4500 open/close in 47.86 seconds: 94.03 ops/second - /unlink 2001 (time 1783141243.55 total 10.00 last 200.05) - /unlink 3768 (time 1783141253.55 total 20.00 last 176.68) total: 4500 /unlink in 23.77 seconds: 189.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1039 (time 1783141270.26 total 10.01 last 103.79) - open/close 2232 (time 1783141280.27 total 20.02 last 119.23) - open/close 3353 (time 1783141290.28 total 30.03 last 111.99) - open/close 4478 (time 1783141300.28 total 40.03 last 112.49) total: 4500 open/close in 40.24 seconds: 111.82 ops/second - /unlink 2103 (time 1783141312.56 total 10.00 last 210.26) - /unlink 4301 (time 1783141322.56 total 20.01 last 219.72) total: 4500 /unlink in 20.99 seconds: 214.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1132 (time 1783141336.52 total 10.00 last 113.15) - open/close 2258 (time 1783141346.52 total 20.00 last 112.60) - open/close 3413 (time 1783141356.53 total 30.01 last 115.40) total: 4500 open/close in 39.56 seconds: 113.75 ops/second - /unlink 2031 (time 1783141378.79 total 10.01 last 202.90) - /unlink 4150 (time 1783141388.79 total 20.01 last 211.82) total: 4500 /unlink in 21.44 seconds: 209.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1069 (time 1783141402.76 total 10.00 last 106.88) - open/close 2163 (time 1783141412.78 total 20.01 last 109.26) - open/close 3266 (time 1783141422.78 total 30.02 last 110.24) - open/close 4434 (time 1783141432.79 total 40.03 last 116.70) total: 4500 open/close in 40.61 seconds: 110.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2076 (time 1783141445.63 total 10.00 last 207.59) - /unlink 4171 (time 1783141455.64 total 20.00 last 209.41) total: 4500 /unlink in 21.36 seconds: 210.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 853 (time 1783141470.32 total 10.01 last 85.24) - open/close 1432 (time 1783141480.32 total 20.01 last 57.90) - open/close 1850 (time 1783141490.64 total 30.33 last 40.49) - open/close 2111 (time 1783141500.73 total 40.42 last 25.87) - open/close 2442 (time 1783141510.73 total 50.43 last 33.08) - open/close 2677 (time 1783141520.76 total 60.46 last 23.43) - open/close 2975 (time 1783141530.80 total 70.49 last 29.70) - open/close 3433 (time 1783141540.80 total 80.49 last 45.79) - open/close 3936 (time 1783141550.81 total 90.50 last 50.23) total: 4500 open/close in 99.16 seconds: 45.38 ops/second - /unlink 1219 (time 1783141574.19 total 10.01 last 121.82) - /unlink 2426 (time 1783141584.19 total 20.01 last 120.67) - /unlink 3781 (time 1783141594.19 total 30.01 last 135.43) total: 4500 /unlink in 35.73 seconds: 125.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 781 (time 1783141613.81 total 10.01 last 78.04) - open/close 1560 (time 1783141623.82 total 20.01 last 77.84) - open/close 2293 (time 1783141633.84 total 30.03 last 73.18) - open/close 3017 (time 1783141643.88 total 40.08 last 72.07) - open/close 3792 (time 1783141653.91 total 50.10 last 77.29) total: 4500 open/close in 59.25 seconds: 75.95 ops/second - /unlink 1630 (time 1783141676.96 total 10.00 last 162.98) - /unlink 3025 (time 1783141686.96 total 20.00 last 139.46) - /unlink 4424 (time 1783141696.97 total 30.02 last 139.74) total: 4500 /unlink in 30.61 seconds: 147.01 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11242: 95081 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server unloading modules via unload_modules_local on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing unload_modules_local oleg124-server: modules unloaded.