== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:56:34 (1788443794) Stopping clients: oleg457-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg457-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg457-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg457-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg457-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg457-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg457-server oleg457-server: oleg457-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: kmemleak disabled oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: MODOPTS_LIBCFS= oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg457-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg457-server: oleg457-server.virtnet: executing set_default_debug -1 all pdsh@oleg457-client: oleg457-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg457-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg457-server: oleg457-server.virtnet: executing set_default_debug -1 all pdsh@oleg457-client: oleg457-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg457-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg457-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg457-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg457-server: oleg457-server.virtnet: executing set_default_debug -1 all pdsh@oleg457-client: oleg457-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg457-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg457-client.virtnet: -o user_xattr,flock 192.168.204.157@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 722 (time 1788443997.09 total 10.00 last 72.17) - open/close 1529 (time 1788444007.09 total 20.01 last 80.68) - open/close 2346 (time 1788444017.09 total 30.01 last 81.70) - open/close 3028 (time 1788444027.10 total 40.02 last 68.11) - open/close 3843 (time 1788444037.12 total 50.04 last 81.35) total: 4500 open/close in 58.31 seconds: 77.18 ops/second - /unlink 1554 (time 1788444060.03 total 10.00 last 155.39) - /unlink 2763 (time 1788444070.04 total 20.01 last 120.76) - /unlink 3764 (time 1788444080.05 total 30.02 last 100.01) total: 4500 /unlink in 36.44 seconds: 123.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 780 (time 1788444100.68 total 10.00 last 77.97) - open/close 1446 (time 1788444110.69 total 20.01 last 66.53) - open/close 2193 (time 1788444120.70 total 30.02 last 74.64) - open/close 2958 (time 1788444130.70 total 40.02 last 76.49) - open/close 3793 (time 1788444140.71 total 50.03 last 83.42) total: 4500 open/close in 58.42 seconds: 77.03 ops/second - /unlink 1559 (time 1788444163.58 total 10.00 last 155.84) - /unlink 3030 (time 1788444173.58 total 20.01 last 147.04) - /unlink 4462 (time 1788444183.59 total 30.01 last 143.14) total: 4500 /unlink in 30.77 seconds: 146.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 667 (time 1788444198.54 total 10.00 last 66.68) - open/close 1385 (time 1788444208.56 total 20.02 last 71.68) - open/close 2201 (time 1788444218.57 total 30.03 last 81.50) - open/close 3005 (time 1788444228.58 total 40.04 last 80.34) - open/close 3820 (time 1788444238.59 total 50.05 last 81.44) total: 4500 open/close in 58.51 seconds: 76.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1447 (time 1788444262.53 total 10.00 last 144.69) - /unlink 2865 (time 1788444272.53 total 20.00 last 141.77) - /unlink 4160 (time 1788444282.53 total 30.00 last 129.47) total: 4500 /unlink in 32.43 seconds: 138.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 737 (time 1788444300.08 total 10.03 last 73.46) - open/close 1427 (time 1788444310.08 total 20.03 last 68.99) - open/close 2009 (time 1788444320.08 total 30.03 last 58.20) - open/close 2726 (time 1788444330.11 total 40.06 last 71.55) - open/close 3484 (time 1788444340.12 total 50.07 last 75.65) - open/close 4230 (time 1788444350.13 total 60.08 last 74.55) total: 4500 open/close in 63.39 seconds: 70.99 ops/second - /unlink 1179 (time 1788444368.45 total 10.00 last 117.85) - /unlink 2381 (time 1788444378.46 total 20.01 last 120.08) - /unlink 3710 (time 1788444388.47 total 30.02 last 132.84) total: 4500 /unlink in 36.33 seconds: 123.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 738 (time 1788444409.31 total 10.00 last 73.77) - open/close 1434 (time 1788444419.32 total 20.01 last 69.55) - open/close 2146 (time 1788444429.33 total 30.03 last 71.10) - open/close 2867 (time 1788444439.34 total 40.03 last 72.04) - open/close 3534 (time 1788444449.35 total 50.04 last 66.64) - open/close 4332 (time 1788444459.35 total 60.05 last 79.75) total: 4500 open/close in 62.43 seconds: 72.08 ops/second - /unlink 1369 (time 1788444476.04 total 10.01 last 136.74) - /unlink 2918 (time 1788444486.04 total 20.01 last 154.85) - /unlink 4250 (time 1788444496.04 total 30.02 last 133.16) total: 4500 /unlink in 32.01 seconds: 140.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 682 (time 1788444512.01 total 10.00 last 68.18) - open/close 1368 (time 1788444522.01 total 20.00 last 68.60) - open/close 2072 (time 1788444532.01 total 30.01 last 70.35) - open/close 2869 (time 1788444542.02 total 40.01 last 79.67) - open/close 3639 (time 1788444552.03 total 50.03 last 76.87) - open/close 4347 (time 1788444562.05 total 60.05 last 70.67) total: 4500 open/close in 62.00 seconds: 72.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1542 (time 1788444579.33 total 10.00 last 154.18) - /unlink 2801 (time 1788444589.34 total 20.01 last 125.80) - /unlink 4176 (time 1788444599.34 total 30.01 last 137.46) total: 4500 /unlink in 32.47 seconds: 138.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 775 (time 1788444616.69 total 10.01 last 77.45) - open/close 1589 (time 1788444626.69 total 20.01 last 81.36) - open/close 2330 (time 1788444636.69 total 30.01 last 74.08) - open/close 3056 (time 1788444646.70 total 40.02 last 72.55) - open/close 3860 (time 1788444656.70 total 50.02 last 80.37) total: 4500 open/close in 58.28 seconds: 77.22 ops/second - /unlink 1432 (time 1788444679.88 total 10.00 last 143.14) - /unlink 2642 (time 1788444689.88 total 20.01 last 120.93) - /unlink 4017 (time 1788444699.89 total 30.01 last 137.49) total: 4500 /unlink in 34.16 seconds: 131.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 775 (time 1788444719.10 total 10.02 last 77.35) - open/close 1572 (time 1788444729.11 total 20.03 last 79.65) - open/close 2368 (time 1788444739.11 total 30.03 last 79.60) - open/close 3141 (time 1788444749.12 total 40.03 last 77.24) - open/close 3909 (time 1788444759.13 total 50.04 last 76.73) total: 4500 open/close in 57.69 seconds: 78.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1527 (time 1788444781.47 total 10.00 last 152.67) - /unlink 2981 (time 1788444791.47 total 20.00 last 145.36) - /unlink 4331 (time 1788444801.49 total 30.03 last 134.70) total: 4500 /unlink in 31.36 seconds: 143.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 750 (time 1788444817.50 total 10.01 last 74.92) - open/close 1460 (time 1788444827.51 total 20.02 last 70.94) - open/close 2090 (time 1788444837.53 total 30.04 last 62.87) - open/close 2802 (time 1788444847.53 total 40.04 last 71.20) - open/close 3497 (time 1788444857.53 total 50.04 last 69.47) - open/close 4247 (time 1788444867.54 total 60.06 last 74.90) total: 4500 open/close in 64.36 seconds: 69.92 ops/second - /unlink 1448 (time 1788444886.34 total 10.01 last 144.64) - /unlink 2924 (time 1788444896.34 total 20.01 last 147.59) - /unlink 4111 (time 1788444906.35 total 30.02 last 118.66) total: 4500 /unlink in 33.11 seconds: 135.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 620 (time 1788444925.14 total 10.05 last 61.72) - open/close 1209 (time 1788444935.16 total 20.06 last 58.83) - open/close 1841 (time 1788444945.17 total 30.07 last 63.10) - open/close 2528 (time 1788444955.18 total 40.08 last 68.67) - open/close 3145 (time 1788444965.18 total 50.08 last 61.65) - open/close 3780 (time 1788444975.19 total 60.09 last 63.48) - open/close 4450 (time 1788444985.19 total 70.09 last 66.95) total: 4500 open/close in 70.70 seconds: 63.65 ops/second - /unlink 1277 (time 1788445000.09 total 10.00 last 127.65) - /unlink 2726 (time 1788445010.09 total 20.01 last 144.86) - /unlink 4009 (time 1788445020.10 total 30.01 last 128.28) total: 4500 /unlink in 34.27 seconds: 131.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 667 (time 1788445039.12 total 10.00 last 66.69) - open/close 1374 (time 1788445049.12 total 20.01 last 70.66) - open/close 2099 (time 1788445059.13 total 30.01 last 72.44) - open/close 2828 (time 1788445069.14 total 40.02 last 72.85) - open/close 3539 (time 1788445079.14 total 50.03 last 71.06) - open/close 4292 (time 1788445089.15 total 60.03 last 75.25) total: 4500 open/close in 62.69 seconds: 71.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1417 (time 1788445106.91 total 10.00 last 141.69) - /unlink 2778 (time 1788445116.91 total 20.00 last 136.05) - /unlink 4165 (time 1788445126.91 total 30.00 last 138.70) total: 4500 /unlink in 32.34 seconds: 139.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 746 (time 1788445144.42 total 10.00 last 74.58) - open/close 1532 (time 1788445154.42 total 20.01 last 78.57) - open/close 2281 (time 1788445164.43 total 30.01 last 74.84) - open/close 2991 (time 1788445174.43 total 40.02 last 70.99) - open/close 3764 (time 1788445184.45 total 50.03 last 77.17) total: 4500 open/close in 59.89 seconds: 75.13 ops/second - /unlink 1514 (time 1788445208.45 total 10.00 last 151.37) - /unlink 2878 (time 1788445218.45 total 20.01 last 136.35) - /unlink 3988 (time 1788445228.46 total 30.01 last 110.96) total: 4500 /unlink in 35.27 seconds: 127.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 703 (time 1788445248.53 total 10.01 last 70.25) - open/close 1436 (time 1788445258.54 total 20.01 last 73.28) - open/close 2196 (time 1788445268.54 total 30.02 last 75.95) - open/close 2988 (time 1788445278.56 total 40.03 last 79.08) - open/close 3789 (time 1788445288.58 total 50.05 last 79.92) total: 4500 open/close in 58.27 seconds: 77.22 ops/second - /unlink 1650 (time 1788445311.11 total 10.00 last 164.93) - /unlink 2941 (time 1788445321.11 total 20.01 last 129.03) - /unlink 4209 (time 1788445331.11 total 30.01 last 126.76) total: 4500 /unlink in 32.22 seconds: 139.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 729 (time 1788445348.33 total 10.01 last 72.80) - open/close 1466 (time 1788445358.34 total 20.03 last 73.61) - open/close 2229 (time 1788445368.35 total 30.03 last 76.29) - open/close 2942 (time 1788445378.36 total 40.04 last 71.20) - open/close 3573 (time 1788445388.36 total 50.04 last 63.09) - open/close 4156 (time 1788445398.37 total 60.05 last 58.25) total: 4500 open/close in 65.18 seconds: 69.04 ops/second - /unlink 1313 (time 1788445418.66 total 10.00 last 131.27) - /unlink 2601 (time 1788445428.66 total 20.00 last 128.78) - /unlink 4084 (time 1788445438.66 total 30.01 last 148.27) total: 4500 /unlink in 33.71 seconds: 133.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 625 (time 1788445458.38 total 10.00 last 62.50) - open/close 1129 (time 1788445468.39 total 20.01 last 50.37) - open/close 1798 (time 1788445478.41 total 30.03 last 66.73) - open/close 2606 (time 1788445488.42 total 40.04 last 80.75) - open/close 3401 (time 1788445498.43 total 50.05 last 79.44) - open/close 4099 (time 1788445508.44 total 60.05 last 69.74) total: 4500 open/close in 65.04 seconds: 69.19 ops/second - /unlink 1534 (time 1788445527.81 total 10.00 last 153.38) - /unlink 3163 (time 1788445537.82 total 20.00 last 162.88) total: 4500 /unlink in 29.24 seconds: 153.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 747 (time 1788445562.67 total 10.00 last 74.68) - open/close 1491 (time 1788445572.68 total 20.01 last 74.32) - open/close 2282 (time 1788445582.69 total 30.02 last 79.08) - open/close 3059 (time 1788445592.70 total 40.03 last 77.62) - open/close 3849 (time 1788445602.70 total 50.03 last 78.98) total: 4500 open/close in 59.13 seconds: 76.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1310 (time 1788445626.45 total 10.01 last 130.83) - /unlink 2530 (time 1788445636.45 total 20.01 last 122.00) - /unlink 3740 (time 1788445646.45 total 30.01 last 121.00) total: 4500 /unlink in 36.20 seconds: 124.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 764 (time 1788445667.66 total 10.03 last 76.13) - open/close 1447 (time 1788445677.67 total 20.04 last 68.25) - open/close 2103 (time 1788445687.68 total 30.06 last 65.51) - open/close 2815 (time 1788445697.68 total 40.06 last 71.18) - open/close 3537 (time 1788445707.70 total 50.08 last 72.06) - open/close 4268 (time 1788445717.72 total 60.09 last 73.01) total: 4500 open/close in 63.52 seconds: 70.84 ops/second - /unlink 1259 (time 1788445735.80 total 10.00 last 125.85) - /unlink 2640 (time 1788445745.80 total 20.01 last 138.08) - /unlink 3927 (time 1788445755.80 total 30.01 last 128.68) total: 4500 /unlink in 33.83 seconds: 133.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 786 (time 1788445775.15 total 10.01 last 78.51) - open/close 1506 (time 1788445785.17 total 20.03 last 71.85) - open/close 2248 (time 1788445795.18 total 30.04 last 74.18) - open/close 2990 (time 1788445805.19 total 40.05 last 74.09) - open/close 3692 (time 1788445815.20 total 50.06 last 70.15) - open/close 4457 (time 1788445825.20 total 60.06 last 76.49) total: 4500 open/close in 60.49 seconds: 74.39 ops/second - /unlink 1296 (time 1788445840.51 total 10.00 last 129.56) - /unlink 2622 (time 1788445850.51 total 20.01 last 132.57) - /unlink 4132 (time 1788445860.51 total 30.01 last 150.98) total: 4500 /unlink in 32.45 seconds: 138.68 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95303 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg457-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg457-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg457-server stop mds service on oleg457-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg457-server stop mds service on oleg457-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg457-server unloading modules via unload_modules_local on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing unload_modules_local oleg457-server: kmemleak disabled modules unloaded.