== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 02:09:35 (1788847775) Stopping clients: oleg420-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg420-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg420-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg420-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg420-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg420-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg420-server oleg420-server: oleg420-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: kmemleak disabled oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: MODOPTS_LIBCFS= oleg420-server: Force libcfs to create 2 CPU partitions oleg420-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg420-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg420-server: oleg420-server.virtnet: executing set_default_debug -1 all pdsh@oleg420-client: oleg420-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg420-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg420-server: oleg420-server.virtnet: executing set_default_debug -1 all pdsh@oleg420-client: oleg420-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg420-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg420-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg420-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg420-server: oleg420-server.virtnet: executing set_default_debug -1 all pdsh@oleg420-client: oleg420-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg420-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg420-client.virtnet: -o user_xattr,flock 192.168.204.120@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 703 (time 1788847988.69 total 10.01 last 70.25) - open/close 1436 (time 1788847998.70 total 20.01 last 73.27) - open/close 2211 (time 1788848008.70 total 30.02 last 77.44) - open/close 2972 (time 1788848018.71 total 40.03 last 76.03) - open/close 3809 (time 1788848028.72 total 50.03 last 83.64) total: 4500 open/close in 59.83 seconds: 75.21 ops/second - /unlink 1106 (time 1788848053.65 total 10.00 last 110.59) - /unlink 2205 (time 1788848063.66 total 20.01 last 109.77) - /unlink 3519 (time 1788848073.67 total 30.02 last 131.34) total: 4500 /unlink in 37.44 seconds: 120.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 629 (time 1788848096.31 total 10.02 last 62.79) - open/close 1282 (time 1788848106.32 total 20.02 last 65.25) - open/close 2058 (time 1788848116.35 total 30.05 last 77.37) - open/close 2742 (time 1788848126.35 total 40.06 last 68.38) - open/close 3392 (time 1788848136.36 total 50.06 last 64.95) - open/close 4152 (time 1788848146.37 total 60.08 last 75.89) total: 4500 open/close in 64.34 seconds: 69.94 ops/second - /unlink 1148 (time 1788848165.15 total 10.01 last 114.72) - /unlink 2559 (time 1788848175.16 total 20.02 last 140.96) - /unlink 3953 (time 1788848185.18 total 30.03 last 139.15) total: 4500 /unlink in 33.37 seconds: 134.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 730 (time 1788848205.83 total 10.00 last 72.99) - open/close 1448 (time 1788848215.86 total 20.04 last 71.56) - open/close 2315 (time 1788848225.86 total 30.04 last 86.70) - open/close 3068 (time 1788848235.87 total 40.04 last 75.25) - open/close 3793 (time 1788848245.87 total 50.04 last 72.50) - open/close 4494 (time 1788848255.87 total 60.05 last 70.07) total: 4500 open/close in 60.09 seconds: 74.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1507 (time 1788848270.50 total 10.01 last 150.58) - /unlink 2943 (time 1788848280.50 total 20.01 last 143.57) - /unlink 4480 (time 1788848290.51 total 30.02 last 153.55) total: 4500 /unlink in 30.22 seconds: 148.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 666 (time 1788848306.65 total 10.01 last 66.56) - open/close 1438 (time 1788848316.65 total 20.01 last 77.19) - open/close 2246 (time 1788848326.66 total 30.02 last 80.72) - open/close 3046 (time 1788848336.67 total 40.02 last 79.94) - open/close 3925 (time 1788848346.67 total 50.02 last 87.88) total: 4500 open/close in 56.35 seconds: 79.85 ops/second - /unlink 1502 (time 1788848368.38 total 10.02 last 149.95) - /unlink 2833 (time 1788848378.41 total 20.04 last 132.75) - /unlink 4380 (time 1788848388.41 total 30.04 last 154.69) total: 4500 /unlink in 30.97 seconds: 145.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 592 (time 1788848404.30 total 10.01 last 59.16) - open/close 1394 (time 1788848414.31 total 20.01 last 80.16) - open/close 2064 (time 1788848424.31 total 30.02 last 66.97) - open/close 2710 (time 1788848434.32 total 40.02 last 64.54) - open/close 3495 (time 1788848444.32 total 50.03 last 78.48) - open/close 4322 (time 1788848454.32 total 60.03 last 82.70) total: 4500 open/close in 62.05 seconds: 72.52 ops/second - /unlink 1534 (time 1788848470.93 total 10.00 last 153.37) - /unlink 2815 (time 1788848480.93 total 20.00 last 128.09) - /unlink 4311 (time 1788848490.93 total 30.01 last 149.56) total: 4500 /unlink in 31.15 seconds: 144.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 563 (time 1788848506.95 total 10.00 last 56.28) - open/close 1114 (time 1788848516.95 total 20.01 last 55.09) - open/close 1719 (time 1788848526.96 total 30.02 last 60.42) - open/close 2374 (time 1788848536.97 total 40.02 last 65.48) - open/close 3175 (time 1788848546.98 total 50.03 last 80.02) - open/close 3616 (time 1788848556.98 total 60.04 last 44.07) - open/close 4407 (time 1788848567.01 total 70.06 last 78.89) total: 4500 open/close in 71.10 seconds: 63.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1575 (time 1788848582.57 total 10.00 last 157.46) - /unlink 3012 (time 1788848592.57 total 20.00 last 143.69) total: 4500 /unlink in 28.90 seconds: 155.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 894 (time 1788848616.11 total 10.01 last 89.31) - open/close 1716 (time 1788848626.11 total 20.02 last 82.16) - open/close 2513 (time 1788848636.12 total 30.02 last 79.68) - open/close 3201 (time 1788848646.13 total 40.03 last 68.73) - open/close 3971 (time 1788848656.13 total 50.03 last 76.96) total: 4500 open/close in 57.27 seconds: 78.58 ops/second - /unlink 1562 (time 1788848678.23 total 10.00 last 156.18) - /unlink 3103 (time 1788848688.23 total 20.00 last 154.06) total: 4500 /unlink in 28.54 seconds: 157.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 887 (time 1788848711.37 total 10.01 last 88.59) - open/close 1751 (time 1788848721.37 total 20.02 last 86.35) - open/close 2575 (time 1788848731.37 total 30.02 last 82.40) - open/close 3366 (time 1788848741.37 total 40.02 last 79.09) - open/close 4121 (time 1788848751.37 total 50.02 last 75.49) total: 4500 open/close in 55.58 seconds: 80.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1477 (time 1788848771.45 total 10.01 last 147.62) - /unlink 2917 (time 1788848781.47 total 20.03 last 143.63) - /unlink 4477 (time 1788848791.48 total 30.04 last 155.93) total: 4500 /unlink in 30.37 seconds: 148.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 806 (time 1788848806.43 total 10.01 last 80.56) - open/close 1657 (time 1788848816.43 total 20.01 last 85.10) - open/close 2448 (time 1788848826.44 total 30.01 last 79.03) - open/close 3210 (time 1788848836.46 total 40.03 last 76.08) - open/close 3996 (time 1788848846.47 total 50.04 last 78.54) total: 4500 open/close in 55.86 seconds: 80.56 ops/second - /unlink 1658 (time 1788848867.15 total 10.00 last 165.77) - /unlink 3043 (time 1788848877.15 total 20.00 last 138.46) - /unlink 4393 (time 1788848887.15 total 30.01 last 134.96) total: 4500 /unlink in 30.94 seconds: 145.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 829 (time 1788848902.14 total 10.00 last 82.89) - open/close 1679 (time 1788848912.15 total 20.01 last 84.94) - open/close 2229 (time 1788848922.17 total 30.02 last 54.91) - open/close 3047 (time 1788848932.17 total 40.03 last 81.75) - open/close 3885 (time 1788848942.17 total 50.03 last 83.79) total: 4500 open/close in 57.46 seconds: 78.31 ops/second - /unlink 1673 (time 1788848963.56 total 10.01 last 167.19) - /unlink 3210 (time 1788848973.56 total 20.01 last 153.69) total: 4500 /unlink in 27.85 seconds: 161.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 850 (time 1788848996.26 total 10.01 last 84.92) - open/close 1711 (time 1788849006.27 total 20.02 last 85.97) - open/close 2544 (time 1788849016.28 total 30.03 last 83.26) - open/close 3464 (time 1788849026.28 total 40.03 last 91.96) - open/close 4243 (time 1788849036.31 total 50.06 last 77.70) total: 4500 open/close in 53.43 seconds: 84.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1543 (time 1788849054.30 total 10.00 last 154.29) - /unlink 3066 (time 1788849064.30 total 20.00 last 152.27) total: 4500 /unlink in 29.14 seconds: 154.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 831 (time 1788849087.75 total 10.01 last 83.05) - open/close 1632 (time 1788849097.76 total 20.01 last 80.04) - open/close 2477 (time 1788849107.76 total 30.02 last 84.43) - open/close 3344 (time 1788849117.77 total 40.03 last 86.64) - open/close 4141 (time 1788849127.78 total 50.03 last 79.67) total: 4500 open/close in 54.93 seconds: 81.93 ops/second - /unlink 1564 (time 1788849146.85 total 10.00 last 156.36) - /unlink 3096 (time 1788849156.85 total 20.00 last 153.18) total: 4500 /unlink in 28.59 seconds: 157.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 862 (time 1788849180.34 total 10.01 last 86.12) - open/close 1544 (time 1788849190.35 total 20.02 last 68.13) - open/close 2296 (time 1788849200.36 total 30.03 last 75.14) - open/close 3137 (time 1788849210.36 total 40.03 last 84.05) - open/close 3750 (time 1788849220.37 total 50.04 last 61.25) total: 4500 open/close in 58.78 seconds: 76.55 ops/second - /unlink 1577 (time 1788849243.55 total 10.00 last 157.64) - /unlink 3192 (time 1788849253.55 total 20.01 last 161.48) total: 4500 /unlink in 28.28 seconds: 159.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 827 (time 1788849276.45 total 10.01 last 82.58) - open/close 1599 (time 1788849286.47 total 20.03 last 77.09) - open/close 2458 (time 1788849296.48 total 30.04 last 85.82) - open/close 3322 (time 1788849306.48 total 40.04 last 86.39) - open/close 4151 (time 1788849316.50 total 50.06 last 82.76) total: 4500 open/close in 54.33 seconds: 82.83 ops/second - /unlink 1424 (time 1788849335.26 total 10.01 last 142.27) - /unlink 2900 (time 1788849345.26 total 20.01 last 147.58) - /unlink 4436 (time 1788849355.26 total 30.01 last 153.59) total: 4500 /unlink in 30.54 seconds: 147.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 794 (time 1788849369.52 total 10.01 last 79.34) - open/close 1603 (time 1788849379.53 total 20.02 last 80.84) - open/close 2423 (time 1788849389.53 total 30.02 last 81.99) - open/close 3264 (time 1788849399.53 total 40.02 last 84.09) - open/close 4113 (time 1788849409.54 total 50.02 last 84.86) total: 4500 open/close in 54.67 seconds: 82.32 ops/second - /unlink 1539 (time 1788849427.72 total 10.00 last 153.86) - /unlink 3056 (time 1788849437.73 total 20.02 last 151.51) total: 4500 /unlink in 29.21 seconds: 154.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 796 (time 1788849460.80 total 10.00 last 79.57) - open/close 1590 (time 1788849470.81 total 20.01 last 79.33) - open/close 2392 (time 1788849480.82 total 30.03 last 80.08) - open/close 3197 (time 1788849490.83 total 40.03 last 80.44) - open/close 4085 (time 1788849500.84 total 50.04 last 88.73) total: 4500 open/close in 54.85 seconds: 82.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1574 (time 1788849519.00 total 10.00 last 157.40) - /unlink 3293 (time 1788849529.00 total 20.00 last 171.85) total: 4500 /unlink in 27.27 seconds: 165.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 913 (time 1788849549.44 total 10.01 last 91.24) - open/close 1786 (time 1788849559.44 total 20.01 last 87.26) - open/close 2597 (time 1788849569.45 total 30.02 last 81.04) - open/close 3416 (time 1788849579.46 total 40.03 last 81.84) - open/close 4227 (time 1788849589.46 total 50.03 last 81.07) total: 4500 open/close in 53.09 seconds: 84.76 ops/second - /unlink 1510 (time 1788849606.14 total 10.00 last 150.99) - /unlink 3066 (time 1788849616.14 total 20.00 last 155.55) total: 4500 /unlink in 29.30 seconds: 153.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 827 (time 1788849639.08 total 10.01 last 82.59) - open/close 1659 (time 1788849649.09 total 20.02 last 83.16) - open/close 2491 (time 1788849659.09 total 30.02 last 83.15) - open/close 3351 (time 1788849669.10 total 40.03 last 85.96) - open/close 4168 (time 1788849679.11 total 50.05 last 81.56) total: 4500 open/close in 53.92 seconds: 83.46 ops/second - /unlink 1660 (time 1788849696.19 total 10.00 last 165.93) - /unlink 3252 (time 1788849706.19 total 20.01 last 159.16) total: 4500 /unlink in 27.91 seconds: 161.22 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95431 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg420-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg420-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg420-server stop mds service on oleg420-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg420-server stop mds service on oleg420-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg420-server unloading modules via unload_modules_local on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing unload_modules_local oleg420-server: kmemleak disabled modules unloaded.