== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:08:24 (1787836104) Stopping clients: oleg608-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg608-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg608-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg608-server oleg608-server: oleg608-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing load_modules_local oleg608-server: Loading modules from /home/green/git/lustre-release/lustre oleg608-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg608-server: detected 4 online CPUs by sysfs oleg608-server: MODOPTS_LIBCFS= oleg608-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg608-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg608-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg608-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg608-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg608-server: oleg608-server.virtnet: executing set_default_debug -1 all pdsh@oleg608-client: oleg608-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg608-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg608-client.virtnet: -o user_xattr,flock 192.168.206.108@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 921 (time 1787836292.64 total 10.01 last 92.05) - open/close 1804 (time 1787836302.65 total 20.01 last 88.27) - open/close 2776 (time 1787836312.65 total 30.01 last 97.19) - open/close 3660 (time 1787836322.65 total 40.01 last 88.39) total: 4500 open/close in 49.82 seconds: 90.32 ops/second - /unlink 1645 (time 1787836346.15 total 10.00 last 164.42) - /unlink 3407 (time 1787836356.16 total 20.01 last 176.13) total: 4500 /unlink in 26.91 seconds: 167.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 847 (time 1787836377.08 total 10.00 last 84.69) - open/close 1584 (time 1787836387.09 total 20.01 last 73.63) - open/close 2245 (time 1787836397.09 total 30.01 last 66.09) - open/close 2900 (time 1787836407.10 total 40.02 last 65.42) - open/close 3723 (time 1787836417.14 total 50.06 last 82.01) total: 4500 open/close in 57.98 seconds: 77.61 ops/second - /unlink 1729 (time 1787836439.16 total 10.01 last 172.73) - /unlink 3420 (time 1787836449.16 total 20.01 last 169.10) total: 4500 /unlink in 27.15 seconds: 165.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 854 (time 1787836471.60 total 10.00 last 85.39) - open/close 1712 (time 1787836481.61 total 20.00 last 85.78) - open/close 2574 (time 1787836491.61 total 30.01 last 86.18) - open/close 3389 (time 1787836501.61 total 40.01 last 81.47) - open/close 4295 (time 1787836511.62 total 50.02 last 90.54) total: 4500 open/close in 52.47 seconds: 85.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1665 (time 1787836528.43 total 10.01 last 166.31) - /unlink 3355 (time 1787836538.44 total 20.02 last 168.85) total: 4500 /unlink in 27.01 seconds: 166.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 904 (time 1787836559.73 total 10.00 last 90.39) - open/close 1849 (time 1787836569.74 total 20.01 last 94.44) - open/close 2774 (time 1787836579.74 total 30.01 last 92.44) - open/close 3684 (time 1787836589.76 total 40.03 last 90.85) total: 4500 open/close in 49.16 seconds: 91.54 ops/second - /unlink 1710 (time 1787836613.03 total 10.00 last 170.92) - /unlink 3506 (time 1787836623.04 total 20.01 last 179.54) total: 4500 /unlink in 26.40 seconds: 170.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 921 (time 1787836643.63 total 10.00 last 92.07) - open/close 1798 (time 1787836653.65 total 20.02 last 87.56) - open/close 2663 (time 1787836663.66 total 30.03 last 86.43) - open/close 3559 (time 1787836673.66 total 40.03 last 89.60) - open/close 4499 (time 1787836683.68 total 50.05 last 93.80) total: 4500 open/close in 50.05 seconds: 89.91 ops/second - /unlink 1682 (time 1787836697.06 total 10.00 last 168.18) - /unlink 3303 (time 1787836707.06 total 20.00 last 162.09) total: 4500 /unlink in 27.27 seconds: 164.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 875 (time 1787836728.62 total 10.00 last 87.46) - open/close 1770 (time 1787836738.63 total 20.01 last 89.47) - open/close 2650 (time 1787836748.63 total 30.01 last 87.96) - open/close 3577 (time 1787836758.64 total 40.02 last 92.62) - open/close 4412 (time 1787836768.64 total 50.02 last 83.49) total: 4500 open/close in 51.14 seconds: 88.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1558 (time 1787836784.14 total 10.00 last 155.74) - /unlink 3167 (time 1787836794.14 total 20.01 last 160.87) total: 4500 /unlink in 28.37 seconds: 158.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 802 (time 1787836817.90 total 10.01 last 80.12) - open/close 1694 (time 1787836827.91 total 20.01 last 89.17) - open/close 2477 (time 1787836837.92 total 30.02 last 78.23) - open/close 3323 (time 1787836847.93 total 40.04 last 84.45) - open/close 4182 (time 1787836857.94 total 50.04 last 85.86) total: 4500 open/close in 54.55 seconds: 82.49 ops/second - /unlink 1441 (time 1787836875.95 total 10.01 last 144.02) - /unlink 2900 (time 1787836885.95 total 20.01 last 145.86) - /unlink 4366 (time 1787836895.96 total 30.01 last 146.56) total: 4500 /unlink in 31.00 seconds: 145.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 859 (time 1787836911.00 total 10.02 last 85.72) - open/close 1613 (time 1787836921.01 total 20.03 last 75.34) - open/close 2398 (time 1787836931.01 total 30.03 last 78.46) - open/close 3320 (time 1787836941.02 total 40.04 last 92.13) - open/close 4184 (time 1787836951.03 total 50.05 last 86.31) total: 4500 open/close in 53.78 seconds: 83.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1599 (time 1787836968.83 total 10.01 last 159.80) - /unlink 3382 (time 1787836978.84 total 20.01 last 178.24) total: 4500 /unlink in 27.36 seconds: 164.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 893 (time 1787836999.84 total 10.01 last 89.25) - open/close 1823 (time 1787837009.85 total 20.01 last 92.92) - open/close 2657 (time 1787837019.86 total 30.02 last 83.34) - open/close 3423 (time 1787837029.87 total 40.03 last 76.53) - open/close 4161 (time 1787837039.87 total 50.04 last 73.77) total: 4500 open/close in 55.29 seconds: 81.39 ops/second - /unlink 1348 (time 1787837059.82 total 10.01 last 134.72) - /unlink 2720 (time 1787837069.82 total 20.01 last 137.16) - /unlink 4055 (time 1787837079.82 total 30.01 last 133.47) total: 4500 /unlink in 32.52 seconds: 138.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 841 (time 1787837097.24 total 10.01 last 84.00) - open/close 1684 (time 1787837107.25 total 20.03 last 84.18) - open/close 2508 (time 1787837117.26 total 30.03 last 82.39) - open/close 3419 (time 1787837127.26 total 40.03 last 91.05) - open/close 4280 (time 1787837137.27 total 50.04 last 86.05) total: 4500 open/close in 52.87 seconds: 85.11 ops/second - /unlink 1414 (time 1787837155.29 total 10.01 last 141.22) - /unlink 3037 (time 1787837165.30 total 20.02 last 162.21) total: 4500 /unlink in 28.92 seconds: 155.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 821 (time 1787837188.84 total 10.00 last 82.06) - open/close 1774 (time 1787837198.84 total 20.01 last 95.26) - open/close 2669 (time 1787837208.87 total 30.03 last 89.31) - open/close 3575 (time 1787837218.88 total 40.04 last 90.47) - open/close 4450 (time 1787837228.88 total 50.05 last 87.46) total: 4500 open/close in 50.54 seconds: 89.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1718 (time 1787837243.03 total 10.00 last 171.77) - /unlink 3436 (time 1787837253.03 total 20.00 last 171.79) total: 4500 /unlink in 26.78 seconds: 168.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 931 (time 1787837273.87 total 10.01 last 93.03) - open/close 1788 (time 1787837283.87 total 20.01 last 85.67) - open/close 2661 (time 1787837293.88 total 30.02 last 87.26) - open/close 3562 (time 1787837303.88 total 40.02 last 90.09) - open/close 4407 (time 1787837313.89 total 50.02 last 84.44) total: 4500 open/close in 51.05 seconds: 88.16 ops/second - /unlink 1649 (time 1787837329.05 total 10.01 last 164.76) - /unlink 3252 (time 1787837339.05 total 20.01 last 160.24) total: 4500 /unlink in 27.55 seconds: 163.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 963 (time 1787837361.46 total 10.00 last 96.29) - open/close 1843 (time 1787837371.47 total 20.01 last 87.94) - open/close 2701 (time 1787837381.48 total 30.02 last 85.71) - open/close 3598 (time 1787837391.49 total 40.03 last 89.62) total: 4500 open/close in 49.62 seconds: 90.69 ops/second - /unlink 1743 (time 1787837414.81 total 10.00 last 174.29) - /unlink 3478 (time 1787837424.81 total 20.00 last 173.44) total: 4500 /unlink in 26.22 seconds: 171.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 868 (time 1787837444.64 total 10.00 last 86.78) - open/close 1753 (time 1787837454.64 total 20.00 last 88.50) - open/close 2610 (time 1787837464.66 total 30.01 last 85.60) - open/close 3459 (time 1787837474.66 total 40.02 last 84.90) - open/close 4340 (time 1787837484.66 total 50.02 last 88.09) total: 4500 open/close in 51.70 seconds: 87.03 ops/second - /unlink 1637 (time 1787837500.00 total 10.01 last 163.56) - /unlink 3289 (time 1787837510.00 total 20.01 last 165.16) total: 4500 /unlink in 27.31 seconds: 164.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 944 (time 1787837531.41 total 10.02 last 94.25) - open/close 1818 (time 1787837541.41 total 20.02 last 87.38) - open/close 2725 (time 1787837551.43 total 30.03 last 90.55) - open/close 3659 (time 1787837561.43 total 40.04 last 93.37) total: 4500 open/close in 49.27 seconds: 91.33 ops/second - /unlink 1616 (time 1787837584.22 total 10.00 last 161.58) - /unlink 3313 (time 1787837594.22 total 20.00 last 169.66) total: 4500 /unlink in 27.17 seconds: 165.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 812 (time 1787837614.37 total 10.02 last 81.01) - open/close 1685 (time 1787837624.38 total 20.04 last 87.15) - open/close 2559 (time 1787837634.39 total 30.05 last 87.36) - open/close 3500 (time 1787837644.40 total 40.06 last 94.02) - open/close 4342 (time 1787837654.40 total 50.06 last 84.15) total: 4500 open/close in 51.84 seconds: 86.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1790 (time 1787837669.79 total 10.00 last 178.93) - /unlink 3410 (time 1787837679.79 total 20.00 last 162.00) total: 4500 /unlink in 26.52 seconds: 169.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 885 (time 1787837700.60 total 10.01 last 88.40) - open/close 1718 (time 1787837710.61 total 20.01 last 83.28) - open/close 2621 (time 1787837720.63 total 30.04 last 90.10) - open/close 3413 (time 1787837730.68 total 40.09 last 78.81) - open/close 4085 (time 1787837740.70 total 50.11 last 67.05) total: 4500 open/close in 54.39 seconds: 82.73 ops/second - /unlink 1634 (time 1787837759.68 total 10.00 last 163.38) - /unlink 3213 (time 1787837769.68 total 20.01 last 157.83) total: 4500 /unlink in 27.70 seconds: 162.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 910 (time 1787837791.95 total 10.01 last 90.91) - open/close 1757 (time 1787837801.97 total 20.03 last 84.56) - open/close 2664 (time 1787837811.97 total 30.03 last 90.67) - open/close 3566 (time 1787837821.98 total 40.04 last 90.14) - open/close 4454 (time 1787837831.98 total 50.04 last 88.75) total: 4500 open/close in 50.52 seconds: 89.08 ops/second - /unlink 1739 (time 1787837845.85 total 10.00 last 173.83) - /unlink 3369 (time 1787837855.85 total 20.01 last 162.96) total: 4500 /unlink in 27.06 seconds: 166.31 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95059 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg608-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg608-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg608-server stop mds service on oleg608-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg608-server unloading modules via unload_modules_local on: 'oleg608-server' oleg608-server: oleg608-server.virtnet: executing unload_modules_local oleg608-server: modules unloaded.