== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:32:38 (1788748358) Stopping clients: oleg449-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg449-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg449-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg449-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg449-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg449-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg449-server oleg449-server: oleg449-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg449-server' oleg449-server: oleg449-server.virtnet: executing load_modules_local oleg449-server: kmemleak disabled oleg449-server: Loading modules from /home/green/git/lustre-release/lustre oleg449-server: detected 4 online CPUs by sysfs oleg449-server: MODOPTS_LIBCFS= oleg449-server: Force libcfs to create 2 CPU partitions oleg449-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg449-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg449-server: oleg449-server.virtnet: executing set_default_debug -1 all pdsh@oleg449-client: oleg449-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg449-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg449-server: oleg449-server.virtnet: executing set_default_debug -1 all pdsh@oleg449-client: oleg449-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg449-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg449-server: oleg449-server.virtnet: executing set_default_debug -1 all pdsh@oleg449-client: oleg449-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg449-client.virtnet: -o user_xattr,flock 192.168.204.149@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 756 (time 1788748559.13 total 10.00 last 75.57) - open/close 1616 (time 1788748569.14 total 20.01 last 85.95) - open/close 2379 (time 1788748579.16 total 30.03 last 76.13) - open/close 3072 (time 1788748589.17 total 40.04 last 69.22) - open/close 3869 (time 1788748599.18 total 50.05 last 79.63) total: 4500 open/close in 58.78 seconds: 76.55 ops/second - /unlink 1404 (time 1788748622.25 total 10.00 last 140.33) - /unlink 2926 (time 1788748632.25 total 20.01 last 152.17) - /unlink 4465 (time 1788748642.25 total 30.01 last 153.88) total: 4500 /unlink in 30.25 seconds: 148.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 739 (time 1788748656.44 total 10.01 last 73.82) - open/close 1574 (time 1788748666.45 total 20.01 last 83.47) - open/close 2424 (time 1788748676.46 total 30.02 last 84.92) - open/close 3225 (time 1788748686.46 total 40.03 last 80.05) - open/close 4065 (time 1788748696.47 total 50.03 last 83.99) total: 4500 open/close in 55.41 seconds: 81.21 ops/second - /unlink 1466 (time 1788748715.81 total 10.00 last 146.54) - /unlink 2861 (time 1788748725.81 total 20.01 last 139.42) - /unlink 4403 (time 1788748735.82 total 30.01 last 154.13) total: 4500 /unlink in 30.93 seconds: 145.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 826 (time 1788748751.39 total 10.01 last 82.51) - open/close 1647 (time 1788748761.39 total 20.01 last 82.09) - open/close 2414 (time 1788748771.39 total 30.01 last 76.70) - open/close 3182 (time 1788748781.40 total 40.02 last 76.72) - open/close 3965 (time 1788748791.41 total 50.03 last 78.22) total: 4500 open/close in 57.44 seconds: 78.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1543 (time 1788748812.97 total 10.01 last 154.14) - /unlink 3094 (time 1788748822.97 total 20.01 last 155.06) total: 4500 /unlink in 29.54 seconds: 152.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 777 (time 1788748847.01 total 10.00 last 77.67) - open/close 1599 (time 1788748857.01 total 20.01 last 82.19) - open/close 2383 (time 1788748867.04 total 30.03 last 78.23) - open/close 3131 (time 1788748877.04 total 40.04 last 74.74) - open/close 3965 (time 1788748887.05 total 50.04 last 83.33) total: 4500 open/close in 56.37 seconds: 79.83 ops/second - /unlink 1504 (time 1788748906.69 total 10.00 last 150.34) - /unlink 3002 (time 1788748916.70 total 20.01 last 149.75) - /unlink 4490 (time 1788748926.70 total 30.01 last 148.69) total: 4500 /unlink in 30.09 seconds: 149.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 836 (time 1788748940.38 total 10.00 last 83.59) - open/close 1637 (time 1788748950.39 total 20.01 last 80.03) - open/close 2403 (time 1788748960.40 total 30.01 last 76.57) - open/close 3220 (time 1788748970.40 total 40.02 last 81.66) - open/close 4011 (time 1788748980.41 total 50.02 last 79.06) total: 4500 open/close in 56.62 seconds: 79.47 ops/second - /unlink 1551 (time 1788749000.72 total 10.00 last 155.08) - /unlink 3136 (time 1788749010.73 total 20.01 last 158.43) total: 4500 /unlink in 28.89 seconds: 155.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 675 (time 1788749033.73 total 10.01 last 67.44) - open/close 1387 (time 1788749043.73 total 20.01 last 71.19) - open/close 2109 (time 1788749053.74 total 30.01 last 72.18) - open/close 2874 (time 1788749063.74 total 40.02 last 76.48) - open/close 3525 (time 1788749073.74 total 50.02 last 65.07) - open/close 4329 (time 1788749083.75 total 60.02 last 80.37) total: 4500 open/close in 62.62 seconds: 71.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1483 (time 1788749100.11 total 10.00 last 148.30) - /unlink 2857 (time 1788749110.11 total 20.00 last 137.37) - /unlink 4221 (time 1788749120.12 total 30.01 last 136.25) total: 4500 /unlink in 32.26 seconds: 139.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 726 (time 1788749136.66 total 10.01 last 72.54) - open/close 1414 (time 1788749146.66 total 20.01 last 68.77) - open/close 2165 (time 1788749156.68 total 30.03 last 74.97) - open/close 2952 (time 1788749166.69 total 40.04 last 78.64) - open/close 3755 (time 1788749176.69 total 50.05 last 80.24) total: 4500 open/close in 59.31 seconds: 75.87 ops/second - /unlink 1508 (time 1788749199.17 total 10.01 last 150.72) - /unlink 3008 (time 1788749209.17 total 20.01 last 149.94) total: 4500 /unlink in 29.94 seconds: 150.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 795 (time 1788749232.73 total 10.01 last 79.43) - open/close 1562 (time 1788749242.74 total 20.02 last 76.60) - open/close 2390 (time 1788749252.74 total 30.03 last 82.75) - open/close 3175 (time 1788749262.75 total 40.03 last 78.45) - open/close 4023 (time 1788749272.76 total 50.04 last 84.72) total: 4500 open/close in 55.70 seconds: 80.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1658 (time 1788749291.48 total 10.01 last 165.69) - /unlink 3456 (time 1788749301.48 total 20.01 last 179.69) total: 4500 /unlink in 25.58 seconds: 175.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 862 (time 1788749319.89 total 10.00 last 86.19) - open/close 1683 (time 1788749329.90 total 20.01 last 82.01) - open/close 2592 (time 1788749339.90 total 30.01 last 90.88) - open/close 3384 (time 1788749349.92 total 40.03 last 79.09) - open/close 4253 (time 1788749359.92 total 50.03 last 86.87) total: 4500 open/close in 53.05 seconds: 84.83 ops/second - /unlink 1571 (time 1788749376.37 total 10.00 last 157.03) - /unlink 3269 (time 1788749386.38 total 20.01 last 169.75) total: 4500 /unlink in 27.92 seconds: 161.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 844 (time 1788749408.28 total 10.00 last 84.36) - open/close 1673 (time 1788749418.29 total 20.02 last 82.77) - open/close 2511 (time 1788749428.30 total 30.02 last 83.78) - open/close 3454 (time 1788749438.30 total 40.03 last 94.23) - open/close 4387 (time 1788749448.30 total 50.03 last 93.29) total: 4500 open/close in 51.17 seconds: 87.95 ops/second - /unlink 1812 (time 1788749462.22 total 10.00 last 181.19) - /unlink 3582 (time 1788749472.22 total 20.00 last 177.00) total: 4500 /unlink in 25.21 seconds: 178.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 876 (time 1788749490.58 total 10.01 last 87.50) - open/close 1727 (time 1788749500.59 total 20.02 last 85.04) - open/close 2696 (time 1788749510.59 total 30.02 last 96.88) - open/close 3719 (time 1788749520.60 total 40.03 last 102.22) total: 4500 open/close in 47.66 seconds: 94.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1230 (time 1788749541.98 total 10.01 last 122.85) - /unlink 2260 (time 1788749551.98 total 20.02 last 102.97) - /unlink 2998 (time 1788749562.00 total 30.03 last 73.71) - /unlink 3749 (time 1788749572.00 total 40.03 last 75.10) total: 4500 /unlink in 46.76 seconds: 96.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 750 (time 1788749592.82 total 10.00 last 74.99) - open/close 1329 (time 1788749602.84 total 20.02 last 57.79) - open/close 1805 (time 1788749612.85 total 30.02 last 47.59) - open/close 2341 (time 1788749622.85 total 40.03 last 53.57) - open/close 3097 (time 1788749632.85 total 50.03 last 75.59) - open/close 3845 (time 1788749642.86 total 60.04 last 74.72) total: 4500 open/close in 68.17 seconds: 66.01 ops/second - /unlink 1543 (time 1788749665.11 total 10.00 last 154.23) - /unlink 3015 (time 1788749675.11 total 20.01 last 147.13) total: 4500 /unlink in 29.83 seconds: 150.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 779 (time 1788749699.19 total 10.02 last 77.76) - open/close 1563 (time 1788749709.19 total 20.02 last 78.40) - open/close 2334 (time 1788749719.21 total 30.04 last 76.95) - open/close 3148 (time 1788749729.22 total 40.05 last 81.33) - open/close 3985 (time 1788749739.22 total 50.05 last 83.68) total: 4500 open/close in 56.66 seconds: 79.43 ops/second - /unlink 1549 (time 1788749759.95 total 10.00 last 154.86) - /unlink 2919 (time 1788749769.96 total 20.01 last 136.95) - /unlink 4246 (time 1788749779.96 total 30.01 last 132.68) total: 4500 /unlink in 31.79 seconds: 141.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 816 (time 1788749796.23 total 10.01 last 81.50) - open/close 1552 (time 1788749806.23 total 20.02 last 73.57) - open/close 2396 (time 1788749816.23 total 30.02 last 84.39) - open/close 3144 (time 1788749826.23 total 40.02 last 74.77) - open/close 3972 (time 1788749836.25 total 50.04 last 82.68) total: 4500 open/close in 57.00 seconds: 78.95 ops/second - /unlink 1397 (time 1788749857.76 total 10.00 last 139.64) - /unlink 2667 (time 1788749867.77 total 20.01 last 126.94) - /unlink 4086 (time 1788749877.78 total 30.02 last 141.76) total: 4500 /unlink in 32.79 seconds: 137.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 774 (time 1788749895.51 total 10.00 last 77.39) - open/close 1511 (time 1788749905.54 total 20.03 last 73.47) - open/close 2303 (time 1788749915.54 total 30.04 last 79.15) - open/close 3065 (time 1788749925.56 total 40.05 last 76.09) - open/close 3826 (time 1788749935.57 total 50.06 last 76.03) total: 4500 open/close in 58.49 seconds: 76.94 ops/second - /unlink 1554 (time 1788749958.95 total 10.00 last 155.37) - /unlink 2929 (time 1788749968.95 total 20.00 last 137.48) - /unlink 4283 (time 1788749978.96 total 30.01 last 135.28) total: 4500 /unlink in 31.67 seconds: 142.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 572 (time 1788749995.77 total 10.01 last 57.15) - open/close 1181 (time 1788750005.77 total 20.01 last 60.86) - open/close 1963 (time 1788750015.78 total 30.02 last 78.12) - open/close 2710 (time 1788750025.79 total 40.03 last 74.65) - open/close 3502 (time 1788750035.79 total 50.03 last 79.18) - open/close 4227 (time 1788750045.80 total 60.04 last 72.45) total: 4500 open/close in 63.55 seconds: 70.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1478 (time 1788750063.55 total 10.00 last 147.80) - /unlink 2891 (time 1788750073.56 total 20.01 last 141.20) - /unlink 4271 (time 1788750083.56 total 30.01 last 137.98) total: 4500 /unlink in 31.82 seconds: 141.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 767 (time 1788750100.20 total 10.00 last 76.67) - open/close 1524 (time 1788750110.21 total 20.01 last 75.65) - open/close 2278 (time 1788750120.21 total 30.01 last 75.38) - open/close 3115 (time 1788750130.22 total 40.02 last 83.65) - open/close 3896 (time 1788750140.22 total 50.02 last 78.08) total: 4500 open/close in 57.31 seconds: 78.52 ops/second - /unlink 1582 (time 1788750161.99 total 10.00 last 158.19) - /unlink 2984 (time 1788750171.99 total 20.00 last 140.18) total: 4500 /unlink in 29.82 seconds: 150.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 718 (time 1788750196.67 total 10.00 last 71.80) - open/close 1386 (time 1788750206.68 total 20.00 last 66.77) - open/close 2057 (time 1788750216.68 total 30.01 last 67.06) - open/close 2633 (time 1788750226.70 total 40.02 last 57.53) - open/close 3198 (time 1788750236.70 total 50.03 last 56.46) - open/close 3932 (time 1788750246.70 total 60.03 last 73.40) total: 4500 open/close in 67.71 seconds: 66.46 ops/second - /unlink 1360 (time 1788750268.89 total 10.00 last 135.99) - /unlink 2786 (time 1788750278.90 total 20.01 last 142.43) - /unlink 4249 (time 1788750288.90 total 30.01 last 146.30) total: 4500 /unlink in 31.75 seconds: 141.75 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95299 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg449-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg449-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg449-server stop mds service on oleg449-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg449-server stop mds service on oleg449-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg449-server unloading modules via unload_modules_local on: 'oleg449-server' oleg449-server: oleg449-server.virtnet: executing unload_modules_local oleg449-server: kmemleak disabled modules unloaded.