== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 11:28:14 (1787153294) Stopping clients: oleg158-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg158-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg158-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg158-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg158-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg158-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg158-server oleg158-server: oleg158-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg158-server' oleg158-server: oleg158-server.virtnet: executing load_modules_local oleg158-server: Loading modules from /home/green/git/lustre-release/lustre oleg158-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg158-server: detected 4 online CPUs by sysfs oleg158-server: MODOPTS_LIBCFS= oleg158-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg158-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg158-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg158-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg158-client.virtnet: -o user_xattr,flock 192.168.201.158@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 808 (time 1787153516.09 total 10.01 last 80.71) - open/close 1549 (time 1787153526.10 total 20.02 last 74.05) - open/close 2374 (time 1787153536.11 total 30.03 last 82.41) - open/close 3197 (time 1787153546.13 total 40.05 last 82.12) - open/close 4038 (time 1787153556.13 total 50.05 last 84.09) total: 4500 open/close in 54.81 seconds: 82.10 ops/second - /unlink 1363 (time 1787153575.73 total 10.01 last 136.14) - /unlink 2921 (time 1787153585.73 total 20.02 last 155.74) - /unlink 4293 (time 1787153595.73 total 30.02 last 137.19) total: 4500 /unlink in 32.16 seconds: 139.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 843 (time 1787153613.08 total 10.02 last 84.10) - open/close 1674 (time 1787153623.08 total 20.03 last 83.08) - open/close 2456 (time 1787153633.09 total 30.03 last 78.15) - open/close 3198 (time 1787153643.09 total 40.04 last 74.15) - open/close 4055 (time 1787153653.10 total 50.05 last 85.64) total: 4500 open/close in 55.81 seconds: 80.63 ops/second - /unlink 1579 (time 1787153674.58 total 10.01 last 157.80) - /unlink 3126 (time 1787153684.58 total 20.01 last 154.69) total: 4500 /unlink in 28.60 seconds: 157.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 663 (time 1787153708.77 total 10.00 last 66.29) - open/close 1408 (time 1787153718.77 total 20.01 last 74.44) - open/close 1993 (time 1787153728.79 total 30.02 last 58.42) - open/close 2808 (time 1787153738.79 total 40.03 last 81.47) - open/close 3623 (time 1787153748.81 total 50.04 last 81.37) total: 4500 open/close in 59.79 seconds: 75.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1326 (time 1787153772.86 total 10.00 last 132.57) - /unlink 2495 (time 1787153782.86 total 20.00 last 116.88) - /unlink 3950 (time 1787153792.86 total 30.01 last 145.46) total: 4500 /unlink in 34.05 seconds: 132.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 565 (time 1787153811.89 total 10.00 last 56.49) - open/close 1213 (time 1787153821.89 total 20.01 last 64.75) - open/close 1905 (time 1787153831.91 total 30.02 last 69.09) - open/close 2466 (time 1787153841.92 total 40.03 last 56.06) - open/close 3039 (time 1787153851.93 total 50.04 last 57.25) - open/close 3614 (time 1787153861.93 total 60.05 last 57.46) - open/close 4242 (time 1787153871.94 total 70.06 last 62.72) total: 4500 open/close in 74.77 seconds: 60.19 ops/second - /unlink 1120 (time 1787153892.60 total 10.00 last 111.99) - /unlink 2239 (time 1787153902.61 total 20.01 last 111.78) - /unlink 3222 (time 1787153912.61 total 30.01 last 98.29) total: 4500 /unlink in 37.38 seconds: 120.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 833 (time 1787153936.09 total 10.00 last 83.30) - open/close 1435 (time 1787153946.09 total 20.00 last 60.20) - open/close 2177 (time 1787153956.12 total 30.03 last 74.01) - open/close 2828 (time 1787153966.13 total 40.03 last 65.05) - open/close 3478 (time 1787153976.13 total 50.04 last 64.98) - open/close 4177 (time 1787153986.13 total 60.04 last 69.89) total: 4500 open/close in 65.11 seconds: 69.12 ops/second - /unlink 1297 (time 1787154006.08 total 10.00 last 129.70) - /unlink 2227 (time 1787154016.08 total 20.00 last 92.96) - /unlink 3356 (time 1787154026.09 total 30.01 last 112.85) - /unlink 4420 (time 1787154036.09 total 40.01 last 106.39) total: 4500 /unlink in 40.63 seconds: 110.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 704 (time 1787154051.43 total 10.01 last 70.33) - open/close 1467 (time 1787154061.44 total 20.02 last 76.21) - open/close 2232 (time 1787154071.46 total 30.04 last 76.37) - open/close 3014 (time 1787154081.47 total 40.05 last 78.11) - open/close 3838 (time 1787154091.47 total 50.06 last 82.36) total: 4500 open/close in 58.29 seconds: 77.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1489 (time 1787154113.79 total 10.01 last 148.76) - /unlink 3204 (time 1787154123.80 total 20.01 last 171.43) total: 4500 /unlink in 27.62 seconds: 162.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 956 (time 1787154145.22 total 10.02 last 95.39) - open/close 1726 (time 1787154155.23 total 20.04 last 76.86) - open/close 2616 (time 1787154165.25 total 30.05 last 88.90) - open/close 3450 (time 1787154175.25 total 40.06 last 83.36) - open/close 4313 (time 1787154185.25 total 50.06 last 86.27) total: 4500 open/close in 52.52 seconds: 85.68 ops/second - /unlink 953 (time 1787154202.47 total 10.00 last 95.30) - /unlink 1974 (time 1787154212.47 total 20.01 last 102.04) - /unlink 3246 (time 1787154222.48 total 30.02 last 127.07) total: 4500 /unlink in 38.19 seconds: 117.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 844 (time 1787154244.84 total 10.02 last 84.26) - open/close 1660 (time 1787154254.87 total 20.05 last 81.33) - open/close 2325 (time 1787154264.90 total 30.07 last 66.37) - open/close 3104 (time 1787154274.90 total 40.07 last 77.89) - open/close 3956 (time 1787154284.90 total 50.08 last 85.14) total: 4500 open/close in 56.66 seconds: 79.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1589 (time 1787154305.73 total 10.00 last 158.87) - /unlink 2640 (time 1787154315.73 total 20.01 last 105.03) - /unlink 3817 (time 1787154325.73 total 30.01 last 117.70) total: 4500 /unlink in 36.72 seconds: 122.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 699 (time 1787154350.03 total 10.00 last 69.89) - open/close 1405 (time 1787154360.04 total 20.01 last 70.54) - open/close 2047 (time 1787154370.04 total 30.01 last 64.18) - open/close 2650 (time 1787154380.04 total 40.01 last 60.29) - open/close 3131 (time 1787154390.05 total 50.03 last 48.03) - open/close 4096 (time 1787154400.08 total 60.05 last 96.27) total: 4500 open/close in 65.04 seconds: 69.19 ops/second - /unlink 1531 (time 1787154418.86 total 10.00 last 153.04) - /unlink 2757 (time 1787154428.86 total 20.01 last 122.57) - /unlink 3816 (time 1787154438.86 total 30.01 last 105.89) total: 4500 /unlink in 34.95 seconds: 128.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 780 (time 1787154457.74 total 10.02 last 77.87) - open/close 1591 (time 1787154467.75 total 20.02 last 81.04) - open/close 2406 (time 1787154477.75 total 30.03 last 81.47) - open/close 3336 (time 1787154487.77 total 40.04 last 92.88) - open/close 4211 (time 1787154497.78 total 50.06 last 87.37) total: 4500 open/close in 53.88 seconds: 83.51 ops/second - /unlink 1355 (time 1787154517.30 total 10.00 last 135.46) - /unlink 2732 (time 1787154527.31 total 20.01 last 137.63) - /unlink 4312 (time 1787154537.32 total 30.02 last 157.82) total: 4500 /unlink in 31.26 seconds: 143.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 776 (time 1787154552.98 total 10.02 last 77.42) - open/close 1470 (time 1787154562.98 total 20.03 last 69.34) - open/close 1937 (time 1787154572.99 total 30.04 last 46.67) - open/close 2610 (time 1787154583.01 total 40.05 last 67.20) - open/close 3437 (time 1787154593.02 total 50.07 last 82.60) - open/close 4398 (time 1787154603.03 total 60.08 last 95.98) total: 4500 open/close in 61.30 seconds: 73.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1439 (time 1787154618.61 total 10.00 last 143.87) - /unlink 2980 (time 1787154628.61 total 20.00 last 154.09) - /unlink 4258 (time 1787154638.61 total 30.00 last 127.79) total: 4500 /unlink in 31.81 seconds: 141.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 850 (time 1787154654.89 total 10.01 last 84.91) - open/close 1774 (time 1787154664.89 total 20.02 last 92.35) - open/close 2594 (time 1787154674.90 total 30.02 last 81.99) - open/close 3409 (time 1787154684.90 total 40.02 last 81.47) - open/close 4268 (time 1787154694.90 total 50.02 last 85.88) total: 4500 open/close in 52.81 seconds: 85.22 ops/second - /unlink 1391 (time 1787154712.26 total 10.00 last 139.09) - /unlink 2982 (time 1787154722.27 total 20.01 last 159.02) total: 4500 /unlink in 29.59 seconds: 152.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 877 (time 1787154746.56 total 10.03 last 87.44) - open/close 1621 (time 1787154756.56 total 20.03 last 74.40) - open/close 2396 (time 1787154766.57 total 30.04 last 77.41) - open/close 3199 (time 1787154776.58 total 40.06 last 80.19) - open/close 4057 (time 1787154786.59 total 50.06 last 85.72) total: 4500 open/close in 55.19 seconds: 81.54 ops/second - /unlink 1645 (time 1787154806.23 total 10.00 last 164.42) - /unlink 2921 (time 1787154816.23 total 20.01 last 127.51) - /unlink 4290 (time 1787154826.24 total 30.02 last 136.84) total: 4500 /unlink in 31.67 seconds: 142.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 960 (time 1787154842.73 total 10.00 last 95.99) - open/close 1848 (time 1787154852.74 total 20.01 last 88.67) - open/close 2794 (time 1787154862.75 total 30.02 last 94.57) - open/close 3654 (time 1787154872.75 total 40.02 last 85.95) - open/close 4441 (time 1787154882.76 total 50.03 last 78.64) total: 4500 open/close in 50.74 seconds: 88.69 ops/second - /unlink 1567 (time 1787154897.72 total 10.01 last 156.50) - /unlink 3155 (time 1787154907.73 total 20.01 last 158.80) total: 4500 /unlink in 28.79 seconds: 156.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 852 (time 1787154931.24 total 10.00 last 85.18) - open/close 1781 (time 1787154941.25 total 20.01 last 92.82) - open/close 2584 (time 1787154951.26 total 30.02 last 80.22) - open/close 3416 (time 1787154961.27 total 40.03 last 83.09) - open/close 4161 (time 1787154971.27 total 50.04 last 74.48) total: 4500 open/close in 53.84 seconds: 83.59 ops/second - /unlink 1726 (time 1787154989.50 total 10.00 last 172.60) - /unlink 3065 (time 1787154999.50 total 20.00 last 133.90) - /unlink 4495 (time 1787155009.53 total 30.03 last 142.64) total: 4500 /unlink in 30.07 seconds: 149.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 635 (time 1787155024.98 total 10.01 last 63.44) - open/close 1353 (time 1787155035.00 total 20.02 last 71.71) - open/close 2299 (time 1787155045.00 total 30.02 last 94.59) - open/close 3140 (time 1787155055.00 total 40.03 last 84.05) - open/close 4063 (time 1787155065.01 total 50.04 last 92.24) total: 4500 open/close in 55.71 seconds: 80.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1725 (time 1787155085.14 total 10.02 last 172.21) - /unlink 3320 (time 1787155095.15 total 20.03 last 159.35) total: 4500 /unlink in 28.60 seconds: 157.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 859 (time 1787155118.99 total 10.01 last 85.84) - open/close 1717 (time 1787155129.01 total 20.02 last 85.65) - open/close 2388 (time 1787155139.02 total 30.04 last 67.01) - open/close 2926 (time 1787155149.02 total 40.04 last 53.80) - open/close 3728 (time 1787155159.03 total 50.05 last 80.12) - open/close 4487 (time 1787155169.04 total 60.06 last 75.81) total: 4500 open/close in 60.28 seconds: 74.65 ops/second - /unlink 1624 (time 1787155184.45 total 10.00 last 162.37) - /unlink 3184 (time 1787155194.45 total 20.00 last 155.98) total: 4500 /unlink in 29.18 seconds: 154.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 812 (time 1787155218.34 total 10.00 last 81.18) - open/close 1754 (time 1787155228.35 total 20.01 last 94.17) - open/close 2584 (time 1787155238.36 total 30.02 last 82.90) - open/close 3499 (time 1787155248.36 total 40.02 last 91.50) total: 4500 open/close in 49.99 seconds: 90.03 ops/second - /unlink 1604 (time 1787155273.71 total 10.00 last 160.37) - /unlink 3133 (time 1787155283.72 total 20.01 last 152.82) total: 4500 /unlink in 29.37 seconds: 153.24 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95288 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg158-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg158-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg158-server stop mds service on oleg158-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg158-server stop mds service on oleg158-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg158-server unloading modules via unload_modules_local on: 'oleg158-server' oleg158-server: oleg158-server.virtnet: executing unload_modules_local oleg158-server: modules unloaded.