== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 12:55:44 (1784048144) Stopping clients: oleg244-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg244-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg244-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg244-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg244-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg244-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg244-server oleg244-server: oleg244-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg244-server' oleg244-server: oleg244-server.virtnet: executing load_modules_local oleg244-server: Loading modules from /home/green/git/lustre-release/lustre oleg244-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg244-server: detected 4 online CPUs by sysfs oleg244-server: MODOPTS_LIBCFS= oleg244-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg244-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg244-server: oleg244-server.virtnet: executing set_default_debug -1 all pdsh@oleg244-client: oleg244-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg244-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg244-server: oleg244-server.virtnet: executing set_default_debug -1 all pdsh@oleg244-client: oleg244-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg244-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg244-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg244-server: oleg244-server.virtnet: executing set_default_debug -1 all pdsh@oleg244-client: oleg244-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg244-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg244-client.virtnet: -o user_xattr,flock 192.168.202.144@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 848 (time 1784048340.33 total 10.01 last 84.71) - open/close 1826 (time 1784048350.33 total 20.02 last 97.75) - open/close 2471 (time 1784048360.34 total 30.02 last 64.45) - open/close 3067 (time 1784048370.39 total 40.07 last 59.30) - open/close 3667 (time 1784048380.40 total 50.08 last 59.94) total: 4500 open/close in 59.81 seconds: 75.24 ops/second - /unlink 1449 (time 1784048404.17 total 10.00 last 144.88) - /unlink 3029 (time 1784048414.18 total 20.00 last 157.95) - /unlink 4406 (time 1784048424.18 total 30.01 last 137.68) total: 4500 /unlink in 30.62 seconds: 146.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 889 (time 1784048438.50 total 10.00 last 88.88) - open/close 1751 (time 1784048448.51 total 20.02 last 86.08) - open/close 2444 (time 1784048458.51 total 30.02 last 69.29) - open/close 3264 (time 1784048468.52 total 40.03 last 81.92) - open/close 4131 (time 1784048478.52 total 50.03 last 86.70) total: 4500 open/close in 53.67 seconds: 83.84 ops/second - /unlink 1468 (time 1784048496.54 total 10.00 last 146.76) - /unlink 3109 (time 1784048506.55 total 20.01 last 164.03) total: 4500 /unlink in 28.63 seconds: 157.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 950 (time 1784048530.54 total 10.01 last 94.93) - open/close 1877 (time 1784048540.54 total 20.01 last 92.70) - open/close 2667 (time 1784048550.56 total 30.03 last 78.84) - open/close 3441 (time 1784048560.56 total 40.03 last 77.39) - open/close 4240 (time 1784048570.57 total 50.04 last 79.80) total: 4500 open/close in 53.24 seconds: 84.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1400 (time 1784048588.81 total 10.00 last 139.99) - /unlink 2997 (time 1784048598.82 total 20.00 last 159.65) total: 4500 /unlink in 28.80 seconds: 156.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 911 (time 1784048622.04 total 10.00 last 91.07) - open/close 1781 (time 1784048632.05 total 20.01 last 86.95) - open/close 2814 (time 1784048642.05 total 30.02 last 103.24) - open/close 3753 (time 1784048652.05 total 40.02 last 93.90) total: 4500 open/close in 48.31 seconds: 93.16 ops/second - /unlink 1704 (time 1784048673.65 total 10.00 last 170.34) - /unlink 3160 (time 1784048683.66 total 20.01 last 145.51) total: 4500 /unlink in 27.44 seconds: 163.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 915 (time 1784048704.63 total 10.01 last 91.42) - open/close 1771 (time 1784048714.64 total 20.01 last 85.59) - open/close 2726 (time 1784048724.64 total 30.01 last 95.49) - open/close 3450 (time 1784048734.64 total 40.02 last 72.37) - open/close 4283 (time 1784048744.64 total 50.02 last 83.28) total: 4500 open/close in 51.96 seconds: 86.60 ops/second - /unlink 1337 (time 1784048760.86 total 10.00 last 133.69) - /unlink 2816 (time 1784048770.87 total 20.00 last 147.86) - /unlink 4348 (time 1784048780.87 total 30.00 last 153.19) total: 4500 /unlink in 31.29 seconds: 143.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 722 (time 1784048796.42 total 10.00 last 72.20) - open/close 1534 (time 1784048806.44 total 20.01 last 81.09) - open/close 2442 (time 1784048816.44 total 30.01 last 90.79) - open/close 3330 (time 1784048826.44 total 40.02 last 88.76) - open/close 4050 (time 1784048836.45 total 50.02 last 71.97) total: 4500 open/close in 55.77 seconds: 80.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1565 (time 1784048856.66 total 10.01 last 156.39) - /unlink 3173 (time 1784048866.66 total 20.01 last 160.79) total: 4500 /unlink in 27.57 seconds: 163.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 852 (time 1784048890.25 total 10.00 last 85.17) - open/close 1718 (time 1784048900.27 total 20.02 last 86.45) - open/close 2555 (time 1784048910.28 total 30.03 last 83.65) - open/close 3346 (time 1784048920.28 total 40.03 last 79.07) - open/close 4306 (time 1784048930.29 total 50.04 last 95.92) total: 4500 open/close in 53.34 seconds: 84.37 ops/second - /unlink 1205 (time 1784048948.30 total 10.00 last 120.49) - /unlink 2594 (time 1784048958.30 total 20.01 last 138.82) - /unlink 3970 (time 1784048968.31 total 30.01 last 137.54) total: 4500 /unlink in 33.58 seconds: 134.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 916 (time 1784048986.59 total 10.00 last 91.59) - open/close 1686 (time 1784048996.60 total 20.01 last 76.92) - open/close 2353 (time 1784049006.61 total 30.02 last 66.61) - open/close 3195 (time 1784049016.62 total 40.04 last 84.10) - open/close 4018 (time 1784049026.63 total 50.04 last 82.26) total: 4500 open/close in 57.03 seconds: 78.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1755 (time 1784049047.84 total 10.01 last 175.32) - /unlink 3294 (time 1784049057.84 total 20.01 last 153.87) total: 4500 /unlink in 27.18 seconds: 165.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 972 (time 1784049079.09 total 10.01 last 97.15) - open/close 1887 (time 1784049089.09 total 20.01 last 91.49) - open/close 2755 (time 1784049099.10 total 30.01 last 86.75) - open/close 3709 (time 1784049109.11 total 40.02 last 95.31) total: 4500 open/close in 48.67 seconds: 92.46 ops/second - /unlink 1644 (time 1784049131.37 total 10.01 last 164.19) - /unlink 3370 (time 1784049141.37 total 20.02 last 172.54) total: 4500 /unlink in 25.73 seconds: 174.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 953 (time 1784049160.69 total 10.00 last 95.27) - open/close 1846 (time 1784049170.69 total 20.01 last 89.25) - open/close 2776 (time 1784049180.70 total 30.01 last 92.98) - open/close 3775 (time 1784049190.70 total 40.01 last 99.89) total: 4500 open/close in 47.44 seconds: 94.86 ops/second - /unlink 1434 (time 1784049213.29 total 10.01 last 143.20) - /unlink 2513 (time 1784049223.29 total 20.02 last 107.85) - /unlink 3727 (time 1784049233.30 total 30.02 last 121.35) total: 4500 /unlink in 35.08 seconds: 128.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 870 (time 1784049253.20 total 10.01 last 86.94) - open/close 1777 (time 1784049263.21 total 20.01 last 90.65) - open/close 2765 (time 1784049273.21 total 30.02 last 98.72) - open/close 3766 (time 1784049283.22 total 40.02 last 100.06) total: 4500 open/close in 47.80 seconds: 94.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1895 (time 1784049304.79 total 10.00 last 189.43) - /unlink 3009 (time 1784049314.79 total 20.01 last 111.37) total: 4500 /unlink in 28.00 seconds: 160.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 884 (time 1784049337.11 total 10.01 last 88.31) - open/close 1773 (time 1784049347.11 total 20.01 last 88.88) - open/close 2713 (time 1784049357.11 total 30.01 last 93.99) - open/close 3799 (time 1784049367.11 total 40.01 last 108.59) total: 4500 open/close in 47.76 seconds: 94.22 ops/second - /unlink 1896 (time 1784049388.74 total 10.00 last 189.58) - /unlink 3981 (time 1784049398.74 total 20.01 last 208.35) total: 4500 /unlink in 23.15 seconds: 194.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 601 (time 1784049418.46 total 10.01 last 60.04) - open/close 1414 (time 1784049428.47 total 20.02 last 81.19) - open/close 2219 (time 1784049438.48 total 30.03 last 80.46) - open/close 3040 (time 1784049448.48 total 40.03 last 82.09) - open/close 3807 (time 1784049458.49 total 50.04 last 76.66) total: 4500 open/close in 59.44 seconds: 75.70 ops/second - /unlink 1428 (time 1784049482.51 total 10.00 last 142.76) - /unlink 2703 (time 1784049492.51 total 20.01 last 127.44) - /unlink 3727 (time 1784049502.52 total 30.01 last 102.34) total: 4500 /unlink in 36.05 seconds: 124.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 915 (time 1784049522.43 total 10.00 last 91.50) - open/close 1711 (time 1784049532.44 total 20.01 last 79.52) - open/close 2546 (time 1784049542.45 total 30.02 last 83.44) - open/close 3295 (time 1784049552.45 total 40.02 last 74.90) - open/close 4181 (time 1784049562.46 total 50.03 last 88.48) total: 4500 open/close in 54.32 seconds: 82.84 ops/second - /unlink 1546 (time 1784049581.46 total 10.00 last 154.59) - /unlink 2894 (time 1784049591.47 total 20.01 last 134.71) - /unlink 4449 (time 1784049601.48 total 30.01 last 155.40) total: 4500 /unlink in 30.46 seconds: 147.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 842 (time 1784049616.34 total 10.00 last 84.17) - open/close 1749 (time 1784049626.35 total 20.01 last 90.65) - open/close 2699 (time 1784049636.37 total 30.03 last 94.82) - open/close 3567 (time 1784049646.37 total 40.03 last 86.75) - open/close 4468 (time 1784049656.38 total 50.04 last 90.06) total: 4500 open/close in 50.50 seconds: 89.11 ops/second - /unlink 1718 (time 1784049671.07 total 10.00 last 171.74) - /unlink 3145 (time 1784049681.08 total 20.00 last 142.69) total: 4500 /unlink in 28.97 seconds: 155.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 828 (time 1784049704.77 total 10.02 last 82.63) - open/close 1420 (time 1784049714.77 total 20.02 last 59.19) - open/close 2077 (time 1784049724.78 total 30.02 last 65.68) - open/close 3017 (time 1784049734.78 total 40.03 last 93.99) - open/close 3849 (time 1784049744.78 total 50.03 last 83.18) total: 4500 open/close in 59.71 seconds: 75.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 708 (time 1784049771.70 total 10.00 last 70.79) - /unlink 2128 (time 1784049781.71 total 20.01 last 141.94) - /unlink 3637 (time 1784049791.72 total 30.02 last 150.61) total: 4500 /unlink in 37.36 seconds: 120.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 668 (time 1784049814.99 total 10.00 last 66.80) - open/close 1483 (time 1784049824.99 total 20.01 last 81.46) - open/close 2096 (time 1784049834.99 total 30.01 last 61.28) - open/close 3050 (time 1784049845.00 total 40.02 last 95.34) - open/close 3964 (time 1784049855.00 total 50.02 last 91.40) total: 4500 open/close in 56.84 seconds: 79.17 ops/second - /unlink 949 (time 1784049878.87 total 10.01 last 94.77) - /unlink 1760 (time 1784049888.88 total 20.02 last 81.02) - /unlink 2485 (time 1784049898.92 total 30.06 last 72.26) - /unlink 3706 (time 1784049908.92 total 40.06 last 122.05) total: 4500 /unlink in 45.97 seconds: 97.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 609 (time 1784049930.59 total 10.00 last 60.89) - open/close 1446 (time 1784049940.60 total 20.01 last 83.64) - open/close 2306 (time 1784049950.60 total 30.01 last 85.98) - open/close 3167 (time 1784049960.63 total 40.05 last 85.80) - open/close 4067 (time 1784049970.63 total 50.05 last 89.98) total: 4500 open/close in 55.19 seconds: 81.53 ops/second - /unlink 1583 (time 1784049990.63 total 10.00 last 158.25) - /unlink 2996 (time 1784050000.63 total 20.00 last 141.28) - /unlink 4482 (time 1784050010.64 total 30.01 last 148.56) total: 4500 /unlink in 30.10 seconds: 149.52 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11243: 95203 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg244-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg244-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg244-server stop mds service on oleg244-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg244-server stop mds service on oleg244-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg244-server unloading modules via unload_modules_local on: 'oleg244-server' oleg244-server: oleg244-server.virtnet: executing unload_modules_local oleg244-server: modules unloaded.