== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 13:00:33 (1788800433) Stopping clients: oleg347-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg347-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg347-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg347-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg347-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg347-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg347-server oleg347-server: oleg347-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg347-server' oleg347-server: oleg347-server.virtnet: executing load_modules_local oleg347-server: kmemleak disabled oleg347-server: Loading modules from /home/green/git/lustre-release/lustre oleg347-server: detected 4 online CPUs by sysfs oleg347-server: MODOPTS_LIBCFS= oleg347-server: Force libcfs to create 2 CPU partitions oleg347-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg347-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg347-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg347-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg347-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg347-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg347-client.virtnet: -o user_xattr,flock 192.168.203.147@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 720 (time 1788800630.77 total 10.00 last 71.98) - open/close 1628 (time 1788800640.78 total 20.01 last 90.74) - open/close 2398 (time 1788800650.79 total 30.02 last 76.94) - open/close 3274 (time 1788800660.80 total 40.03 last 87.52) - open/close 3979 (time 1788800670.81 total 50.03 last 70.44) total: 4500 open/close in 58.18 seconds: 77.35 ops/second - /unlink 1422 (time 1788800693.59 total 10.00 last 142.15) - /unlink 2709 (time 1788800703.60 total 20.02 last 128.54) - /unlink 4097 (time 1788800713.61 total 30.03 last 138.64) total: 4500 /unlink in 32.63 seconds: 137.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 828 (time 1788800730.37 total 10.00 last 82.78) - open/close 1599 (time 1788800740.37 total 20.01 last 77.07) - open/close 2336 (time 1788800750.38 total 30.01 last 73.64) - open/close 3157 (time 1788800760.38 total 40.02 last 82.07) - open/close 3865 (time 1788800770.38 total 50.02 last 70.78) total: 4500 open/close in 57.46 seconds: 78.32 ops/second - /unlink 1575 (time 1788800791.97 total 10.01 last 157.38) - /unlink 3134 (time 1788800801.97 total 20.01 last 155.89) total: 4500 /unlink in 29.01 seconds: 155.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 825 (time 1788800825.67 total 10.01 last 82.46) - open/close 1557 (time 1788800835.67 total 20.01 last 73.15) - open/close 2418 (time 1788800845.68 total 30.01 last 86.08) - open/close 3232 (time 1788800855.68 total 40.02 last 81.33) - open/close 3952 (time 1788800865.70 total 50.04 last 71.88) total: 4500 open/close in 58.67 seconds: 76.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1303 (time 1788800889.11 total 10.00 last 130.28) - /unlink 2556 (time 1788800899.11 total 20.01 last 125.23) - /unlink 3845 (time 1788800909.11 total 30.01 last 128.90) total: 4500 /unlink in 34.93 seconds: 128.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 703 (time 1788800928.18 total 10.01 last 70.24) - open/close 1465 (time 1788800938.18 total 20.01 last 76.18) - open/close 2235 (time 1788800948.19 total 30.02 last 76.94) - open/close 2999 (time 1788800958.21 total 40.04 last 76.26) - open/close 3721 (time 1788800968.22 total 50.04 last 72.14) - open/close 4497 (time 1788800978.22 total 60.05 last 77.59) total: 4500 open/close in 60.07 seconds: 74.91 ops/second - /unlink 1255 (time 1788800992.57 total 10.00 last 125.49) - /unlink 2594 (time 1788801002.58 total 20.01 last 133.81) - /unlink 3913 (time 1788801012.58 total 30.01 last 131.84) total: 4500 /unlink in 33.93 seconds: 132.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 853 (time 1788801031.07 total 10.02 last 85.12) - open/close 1628 (time 1788801041.08 total 20.03 last 77.43) - open/close 2368 (time 1788801051.09 total 30.05 last 73.89) - open/close 3191 (time 1788801061.10 total 40.05 last 82.23) - open/close 4045 (time 1788801071.11 total 50.06 last 85.35) total: 4500 open/close in 57.30 seconds: 78.54 ops/second - /unlink 1606 (time 1788801092.79 total 10.00 last 160.53) - /unlink 3183 (time 1788801102.80 total 20.01 last 157.56) total: 4500 /unlink in 28.49 seconds: 157.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 752 (time 1788801125.93 total 10.01 last 75.15) - open/close 1591 (time 1788801135.93 total 20.01 last 83.88) - open/close 2417 (time 1788801145.93 total 30.02 last 82.55) - open/close 3288 (time 1788801155.95 total 40.03 last 86.97) - open/close 4053 (time 1788801165.95 total 50.03 last 76.48) total: 4500 open/close in 55.57 seconds: 80.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1489 (time 1788801185.73 total 10.00 last 148.85) - /unlink 3063 (time 1788801195.74 total 20.01 last 157.36) - /unlink 4355 (time 1788801205.75 total 30.01 last 129.09) total: 4500 /unlink in 31.50 seconds: 142.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 777 (time 1788801221.54 total 10.00 last 77.67) - open/close 1658 (time 1788801231.55 total 20.02 last 87.97) - open/close 2244 (time 1788801241.57 total 30.03 last 58.53) - open/close 2967 (time 1788801251.57 total 40.03 last 72.29) - open/close 3794 (time 1788801261.57 total 50.04 last 82.65) total: 4500 open/close in 58.67 seconds: 76.70 ops/second - /unlink 1570 (time 1788801284.22 total 10.00 last 156.97) - /unlink 3192 (time 1788801294.22 total 20.00 last 162.18) total: 4500 /unlink in 28.12 seconds: 160.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 811 (time 1788801316.53 total 10.01 last 81.00) - open/close 1600 (time 1788801326.55 total 20.03 last 78.79) - open/close 2373 (time 1788801336.56 total 30.03 last 77.24) - open/close 3187 (time 1788801346.56 total 40.04 last 81.36) - open/close 3737 (time 1788801356.56 total 50.04 last 55.00) - open/close 4361 (time 1788801366.57 total 60.05 last 62.36) total: 4500 open/close in 62.92 seconds: 71.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1595 (time 1788801383.87 total 10.00 last 159.44) - /unlink 3109 (time 1788801393.87 total 20.00 last 151.39) total: 4500 /unlink in 29.16 seconds: 154.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 773 (time 1788801417.72 total 10.00 last 77.29) - open/close 1555 (time 1788801427.72 total 20.00 last 78.18) - open/close 2322 (time 1788801437.74 total 30.02 last 76.56) - open/close 3137 (time 1788801447.75 total 40.03 last 81.41) - open/close 3929 (time 1788801457.75 total 50.04 last 79.15) total: 4500 open/close in 57.16 seconds: 78.73 ops/second - /unlink 1540 (time 1788801479.56 total 10.00 last 153.96) - /unlink 3000 (time 1788801489.57 total 20.01 last 145.96) - /unlink 4431 (time 1788801499.57 total 30.01 last 143.04) total: 4500 /unlink in 30.67 seconds: 146.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 749 (time 1788801515.22 total 10.02 last 74.79) - open/close 1527 (time 1788801525.23 total 20.03 last 77.70) - open/close 2246 (time 1788801535.24 total 30.03 last 71.85) - open/close 2947 (time 1788801545.25 total 40.05 last 70.00) - open/close 3680 (time 1788801555.26 total 50.05 last 73.26) total: 4500 open/close in 59.67 seconds: 75.41 ops/second - /unlink 1641 (time 1788801578.57 total 10.00 last 164.07) - /unlink 3268 (time 1788801588.57 total 20.01 last 162.64) total: 4500 /unlink in 27.93 seconds: 161.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 841 (time 1788801610.66 total 10.00 last 84.10) - open/close 1690 (time 1788801620.66 total 20.00 last 84.90) - open/close 2526 (time 1788801630.66 total 30.00 last 83.59) - open/close 3225 (time 1788801640.67 total 40.02 last 69.81) - open/close 4071 (time 1788801650.68 total 50.02 last 84.54) total: 4500 open/close in 55.46 seconds: 81.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1476 (time 1788801670.59 total 10.00 last 147.58) - /unlink 2952 (time 1788801680.59 total 20.01 last 147.52) - /unlink 4291 (time 1788801690.60 total 30.01 last 133.83) total: 4500 /unlink in 31.33 seconds: 143.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 627 (time 1788801706.51 total 10.01 last 62.66) - open/close 1373 (time 1788801716.51 total 20.01 last 74.57) - open/close 2046 (time 1788801726.52 total 30.02 last 67.24) - open/close 2859 (time 1788801736.53 total 40.03 last 81.24) - open/close 3674 (time 1788801746.53 total 50.03 last 81.47) total: 4500 open/close in 59.14 seconds: 76.09 ops/second - /unlink 1514 (time 1788801769.18 total 10.01 last 151.28) - /unlink 2993 (time 1788801779.18 total 20.01 last 147.89) - /unlink 4140 (time 1788801789.18 total 30.01 last 114.67) total: 4500 /unlink in 33.61 seconds: 133.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 794 (time 1788801807.49 total 10.01 last 79.29) - open/close 1558 (time 1788801817.49 total 20.01 last 76.40) - open/close 2354 (time 1788801827.50 total 30.02 last 79.57) - open/close 3189 (time 1788801837.51 total 40.02 last 83.45) - open/close 3923 (time 1788801847.53 total 50.05 last 73.19) total: 4500 open/close in 57.49 seconds: 78.27 ops/second - /unlink 1429 (time 1788801868.70 total 10.00 last 142.85) - /unlink 2753 (time 1788801878.70 total 20.00 last 132.40) - /unlink 4194 (time 1788801888.70 total 30.01 last 144.08) total: 4500 /unlink in 31.90 seconds: 141.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 688 (time 1788801904.36 total 10.00 last 68.79) - open/close 1551 (time 1788801914.37 total 20.00 last 86.27) - open/close 2388 (time 1788801924.37 total 30.01 last 83.67) - open/close 3182 (time 1788801934.37 total 40.01 last 79.38) - open/close 3797 (time 1788801944.38 total 50.02 last 61.44) - open/close 4336 (time 1788801954.39 total 60.03 last 53.84) total: 4500 open/close in 63.59 seconds: 70.76 ops/second - /unlink 1450 (time 1788801973.07 total 10.00 last 144.96) - /unlink 2916 (time 1788801983.08 total 20.01 last 146.54) - /unlink 4059 (time 1788801993.08 total 30.01 last 114.29) total: 4500 /unlink in 33.54 seconds: 134.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 809 (time 1788802011.39 total 10.00 last 80.89) - open/close 1673 (time 1788802021.39 total 20.01 last 86.35) - open/close 2557 (time 1788802031.41 total 30.02 last 88.29) - open/close 3379 (time 1788802041.41 total 40.03 last 82.14) - open/close 4257 (time 1788802051.42 total 50.04 last 87.72) total: 4500 open/close in 53.14 seconds: 84.68 ops/second - /unlink 1646 (time 1788802068.55 total 10.01 last 164.46) - /unlink 3195 (time 1788802078.56 total 20.02 last 154.69) total: 4500 /unlink in 28.13 seconds: 159.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 866 (time 1788802101.07 total 10.01 last 86.53) - open/close 1613 (time 1788802111.07 total 20.01 last 74.70) - open/close 2477 (time 1788802121.07 total 30.01 last 86.37) - open/close 3287 (time 1788802131.08 total 40.02 last 80.94) - open/close 4027 (time 1788802141.08 total 50.02 last 73.96) total: 4500 open/close in 55.29 seconds: 81.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1475 (time 1788802160.43 total 10.00 last 147.48) - /unlink 3141 (time 1788802170.43 total 20.00 last 166.57) total: 4500 /unlink in 29.37 seconds: 153.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 857 (time 1788802194.29 total 10.00 last 85.67) - open/close 1673 (time 1788802204.29 total 20.01 last 81.56) - open/close 2451 (time 1788802214.30 total 30.01 last 77.79) - open/close 3161 (time 1788802224.31 total 40.02 last 70.89) - open/close 3827 (time 1788802234.32 total 50.03 last 66.56) total: 4500 open/close in 59.30 seconds: 75.88 ops/second - /unlink 1403 (time 1788802258.62 total 10.00 last 140.29) - /unlink 2884 (time 1788802268.63 total 20.01 last 148.00) - /unlink 4312 (time 1788802278.64 total 30.01 last 142.69) total: 4500 /unlink in 31.23 seconds: 144.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 819 (time 1788802294.56 total 10.01 last 81.83) - open/close 1591 (time 1788802304.57 total 20.01 last 77.17) - open/close 2477 (time 1788802314.58 total 30.02 last 88.49) - open/close 3340 (time 1788802324.58 total 40.03 last 86.24) - open/close 4153 (time 1788802334.59 total 50.03 last 81.28) total: 4500 open/close in 54.01 seconds: 83.32 ops/second - /unlink 1565 (time 1788802352.25 total 10.00 last 156.47) - /unlink 3139 (time 1788802362.26 total 20.01 last 157.30) total: 4500 /unlink in 29.54 seconds: 152.36 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95129 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg347-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg347-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg347-server stop mds service on oleg347-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg347-server stop mds service on oleg347-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg347-server unloading modules via unload_modules_local on: 'oleg347-server' oleg347-server: oleg347-server.virtnet: executing unload_modules_local oleg347-server: kmemleak disabled modules unloaded.