== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 11:35:19 (1787844919) Stopping clients: oleg245-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg245-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg245-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg245-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg245-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg245-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg245-server oleg245-server: oleg245-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg245-server' oleg245-server: oleg245-server.virtnet: executing load_modules_local oleg245-server: kmemleak disabled oleg245-server: Loading modules from /home/green/git/lustre-release/lustre oleg245-server: detected 4 online CPUs by sysfs oleg245-server: MODOPTS_LIBCFS= oleg245-server: Force libcfs to create 2 CPU partitions oleg245-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg245-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg245-server: oleg245-server.virtnet: executing set_default_debug -1 all pdsh@oleg245-client: oleg245-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg245-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg245-server: oleg245-server.virtnet: executing set_default_debug -1 all pdsh@oleg245-client: oleg245-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg245-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg245-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg245-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg245-server: oleg245-server.virtnet: executing set_default_debug -1 all pdsh@oleg245-client: oleg245-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg245-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg245-client.virtnet: -o user_xattr,flock 192.168.202.145@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 708 (time 1787845143.89 total 10.00 last 70.79) - open/close 1343 (time 1787845153.90 total 20.01 last 63.43) - open/close 1996 (time 1787845163.92 total 30.04 last 65.15) - open/close 2535 (time 1787845173.95 total 40.06 last 53.77) - open/close 3245 (time 1787845183.95 total 50.06 last 70.98) - open/close 4143 (time 1787845193.97 total 60.08 last 89.66) total: 4500 open/close in 65.79 seconds: 68.39 ops/second - /unlink 1227 (time 1787845214.61 total 10.01 last 122.64) - /unlink 2734 (time 1787845224.61 total 20.01 last 150.68) - /unlink 4307 (time 1787845234.62 total 30.01 last 157.23) total: 4500 /unlink in 31.76 seconds: 141.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 899 (time 1787845251.97 total 10.00 last 89.89) - open/close 1773 (time 1787845261.98 total 20.02 last 87.26) - open/close 2697 (time 1787845271.98 total 30.02 last 92.39) - open/close 3658 (time 1787845281.99 total 40.03 last 96.02) - open/close 4341 (time 1787845292.00 total 50.04 last 68.24) total: 4500 open/close in 52.00 seconds: 86.54 ops/second - /unlink 1599 (time 1787845308.77 total 10.00 last 159.88) - /unlink 3041 (time 1787845318.77 total 20.01 last 144.11) total: 4500 /unlink in 29.85 seconds: 150.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 892 (time 1787845343.88 total 10.00 last 89.19) - open/close 1810 (time 1787845353.89 total 20.01 last 91.70) - open/close 2737 (time 1787845363.90 total 30.02 last 92.60) - open/close 3532 (time 1787845373.90 total 40.03 last 79.47) - open/close 4410 (time 1787845383.91 total 50.04 last 87.73) total: 4500 open/close in 50.85 seconds: 88.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1475 (time 1787845398.62 total 10.00 last 147.44) - /unlink 3081 (time 1787845408.62 total 20.01 last 160.53) total: 4500 /unlink in 28.04 seconds: 160.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 832 (time 1787845431.52 total 10.01 last 83.11) - open/close 1784 (time 1787845441.53 total 20.01 last 95.17) - open/close 2735 (time 1787845451.53 total 30.02 last 95.09) - open/close 3471 (time 1787845461.53 total 40.02 last 73.57) - open/close 4318 (time 1787845471.54 total 50.02 last 84.66) total: 4500 open/close in 53.71 seconds: 83.78 ops/second - /unlink 1734 (time 1787845491.90 total 10.04 last 172.71) - /unlink 3203 (time 1787845501.91 total 20.04 last 146.87) total: 4500 /unlink in 28.00 seconds: 160.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 791 (time 1787845524.83 total 10.00 last 79.08) - open/close 1563 (time 1787845534.84 total 20.01 last 77.17) - open/close 2352 (time 1787845544.84 total 30.02 last 78.83) - open/close 3192 (time 1787845554.85 total 40.02 last 83.95) - open/close 4155 (time 1787845564.86 total 50.03 last 96.23) total: 4500 open/close in 53.96 seconds: 83.40 ops/second - /unlink 1566 (time 1787845583.30 total 10.00 last 156.57) - /unlink 3130 (time 1787845593.30 total 20.00 last 156.37) total: 4500 /unlink in 29.04 seconds: 154.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 882 (time 1787845617.50 total 10.00 last 88.19) - open/close 1727 (time 1787845627.52 total 20.02 last 84.37) - open/close 2546 (time 1787845637.52 total 30.02 last 81.86) - open/close 3514 (time 1787845647.53 total 40.02 last 96.79) - open/close 4400 (time 1787845657.53 total 50.02 last 88.59) total: 4500 open/close in 50.98 seconds: 88.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1597 (time 1787845673.27 total 10.00 last 159.66) - /unlink 3029 (time 1787845683.27 total 20.00 last 143.17) - /unlink 4289 (time 1787845693.27 total 30.01 last 125.94) total: 4500 /unlink in 31.84 seconds: 141.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 594 (time 1787845711.27 total 10.02 last 59.31) - open/close 1194 (time 1787845721.28 total 20.03 last 59.94) - open/close 2160 (time 1787845731.28 total 30.03 last 96.57) - open/close 2994 (time 1787845741.29 total 40.04 last 83.34) - open/close 3769 (time 1787845751.30 total 50.06 last 77.35) total: 4500 open/close in 59.95 seconds: 75.07 ops/second - /unlink 1600 (time 1787845775.76 total 10.00 last 159.94) - /unlink 3200 (time 1787845785.77 total 20.01 last 159.91) total: 4500 /unlink in 28.60 seconds: 157.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 519 (time 1787845810.58 total 10.00 last 51.89) - open/close 1149 (time 1787845820.58 total 20.01 last 62.95) - open/close 1649 (time 1787845830.63 total 30.05 last 49.80) - open/close 2206 (time 1787845840.63 total 40.06 last 55.65) - open/close 3073 (time 1787845850.64 total 50.07 last 86.65) - open/close 3769 (time 1787845860.65 total 60.07 last 69.54) total: 4500 open/close in 69.55 seconds: 64.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1431 (time 1787845885.47 total 10.00 last 143.04) - /unlink 2591 (time 1787845895.48 total 20.01 last 115.93) - /unlink 3707 (time 1787845905.48 total 30.01 last 111.59) total: 4500 /unlink in 35.80 seconds: 125.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 689 (time 1787845926.36 total 10.02 last 68.79) - open/close 1354 (time 1787845936.38 total 20.04 last 66.36) - open/close 1985 (time 1787845946.41 total 30.07 last 62.90) - open/close 2823 (time 1787845956.42 total 40.07 last 83.74) - open/close 3571 (time 1787845966.42 total 50.08 last 74.79) - open/close 4352 (time 1787845976.44 total 60.10 last 77.89) total: 4500 open/close in 62.08 seconds: 72.48 ops/second - /unlink 1775 (time 1787845992.68 total 10.00 last 177.49) - /unlink 3359 (time 1787846002.69 total 20.02 last 158.13) total: 4500 /unlink in 28.55 seconds: 157.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 855 (time 1787846026.23 total 10.01 last 85.43) - open/close 1636 (time 1787846036.23 total 20.01 last 78.10) - open/close 2395 (time 1787846046.23 total 30.02 last 75.83) - open/close 3182 (time 1787846056.25 total 40.03 last 78.59) - open/close 3997 (time 1787846066.25 total 50.04 last 81.45) total: 4500 open/close in 55.78 seconds: 80.67 ops/second - /unlink 1776 (time 1787846086.61 total 10.01 last 177.48) - /unlink 3159 (time 1787846096.63 total 20.03 last 138.00) total: 4500 /unlink in 29.96 seconds: 150.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 723 (time 1787846121.33 total 10.03 last 72.12) - open/close 1556 (time 1787846131.34 total 20.03 last 83.27) - open/close 2373 (time 1787846141.35 total 30.04 last 81.60) - open/close 3369 (time 1787846151.36 total 40.05 last 99.54) - open/close 4066 (time 1787846161.38 total 50.07 last 69.55) total: 4500 open/close in 55.22 seconds: 81.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1571 (time 1787846181.63 total 10.01 last 156.97) - /unlink 3256 (time 1787846191.64 total 20.01 last 168.40) total: 4500 /unlink in 28.26 seconds: 159.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 802 (time 1787846215.44 total 10.00 last 80.17) - open/close 1540 (time 1787846225.44 total 20.01 last 73.76) - open/close 2455 (time 1787846235.44 total 30.01 last 91.47) - open/close 3284 (time 1787846245.45 total 40.02 last 82.84) - open/close 3971 (time 1787846255.48 total 50.05 last 68.51) total: 4500 open/close in 57.59 seconds: 78.14 ops/second - /unlink 1379 (time 1787846277.84 total 10.00 last 137.84) - /unlink 2891 (time 1787846287.84 total 20.00 last 151.20) total: 4500 /unlink in 29.73 seconds: 151.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 563 (time 1787846312.65 total 10.01 last 56.24) - open/close 1181 (time 1787846322.65 total 20.01 last 61.79) - open/close 1957 (time 1787846332.66 total 30.02 last 77.56) - open/close 2621 (time 1787846342.67 total 40.02 last 66.35) - open/close 3227 (time 1787846352.67 total 50.03 last 60.56) - open/close 3842 (time 1787846362.68 total 60.04 last 61.44) total: 4500 open/close in 69.27 seconds: 64.96 ops/second - /unlink 1487 (time 1787846387.06 total 10.00 last 148.66) - /unlink 3021 (time 1787846397.06 total 20.00 last 153.38) - /unlink 4494 (time 1787846407.08 total 30.02 last 147.08) total: 4500 /unlink in 30.05 seconds: 149.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 641 (time 1787846423.04 total 10.01 last 64.04) - open/close 1274 (time 1787846433.05 total 20.01 last 63.28) - open/close 2028 (time 1787846443.05 total 30.01 last 75.40) - open/close 2860 (time 1787846453.05 total 40.01 last 83.20) - open/close 3694 (time 1787846463.05 total 50.02 last 83.37) - open/close 4434 (time 1787846473.05 total 60.02 last 73.97) total: 4500 open/close in 60.69 seconds: 74.15 ops/second - /unlink 1432 (time 1787846488.73 total 10.03 last 142.80) - /unlink 2724 (time 1787846498.73 total 20.03 last 129.16) - /unlink 4444 (time 1787846508.73 total 30.03 last 171.94) total: 4500 /unlink in 30.82 seconds: 146.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 948 (time 1787846524.85 total 10.00 last 94.77) - open/close 1957 (time 1787846534.86 total 20.01 last 100.83) - open/close 2838 (time 1787846544.88 total 30.03 last 87.91) - open/close 3536 (time 1787846554.89 total 40.04 last 69.74) - open/close 4330 (time 1787846564.89 total 50.05 last 79.37) total: 4500 open/close in 51.78 seconds: 86.90 ops/second - /unlink 1204 (time 1787846581.42 total 10.00 last 120.37) - /unlink 2475 (time 1787846591.43 total 20.01 last 126.96) - /unlink 3722 (time 1787846601.44 total 30.02 last 124.57) total: 4500 /unlink in 36.99 seconds: 121.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 921 (time 1787846624.47 total 10.00 last 92.09) - open/close 1894 (time 1787846634.47 total 20.00 last 97.30) - open/close 2779 (time 1787846644.48 total 30.01 last 88.44) - open/close 3652 (time 1787846654.50 total 40.03 last 87.12) - open/close 4467 (time 1787846664.50 total 50.04 last 81.43) total: 4500 open/close in 50.45 seconds: 89.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1325 (time 1787846679.66 total 10.00 last 132.48) - /unlink 2503 (time 1787846689.66 total 20.00 last 117.77) - /unlink 4193 (time 1787846699.66 total 30.01 last 168.97) total: 4500 /unlink in 32.01 seconds: 140.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 952 (time 1787846716.73 total 10.01 last 95.09) - open/close 1741 (time 1787846726.73 total 20.01 last 78.89) - open/close 2613 (time 1787846736.74 total 30.02 last 87.13) - open/close 3435 (time 1787846746.77 total 40.05 last 81.96) - open/close 4295 (time 1787846756.77 total 50.06 last 85.93) total: 4500 open/close in 53.04 seconds: 84.83 ops/second - /unlink 1505 (time 1787846775.02 total 10.00 last 150.50) - /unlink 2943 (time 1787846785.03 total 20.01 last 143.68) - /unlink 4204 (time 1787846795.03 total 30.01 last 126.05) total: 4500 /unlink in 31.96 seconds: 140.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 562 (time 1787846812.23 total 10.01 last 56.13) - open/close 1189 (time 1787846822.23 total 20.02 last 62.68) - open/close 2092 (time 1787846832.24 total 30.02 last 90.23) - open/close 2969 (time 1787846842.24 total 40.02 last 87.69) - open/close 3739 (time 1787846852.24 total 50.03 last 76.97) total: 4500 open/close in 58.07 seconds: 77.49 ops/second - /unlink 1574 (time 1787846874.50 total 10.00 last 157.40) - /unlink 3201 (time 1787846884.50 total 20.00 last 162.67) total: 4500 /unlink in 28.60 seconds: 157.32 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95307 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg245-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg245-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg245-server stop mds service on oleg245-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg245-server stop mds service on oleg245-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg245-server unloading modules via unload_modules_local on: 'oleg245-server' oleg245-server: oleg245-server.virtnet: executing unload_modules_local oleg245-server: kmemleak disabled modules unloaded.