== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:45:18 (1788756318) Stopping clients: oleg353-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg353-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg353-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg353-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg353-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg353-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg353-server oleg353-server: oleg353-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg353-server' oleg353-server: oleg353-server.virtnet: executing load_modules_local oleg353-server: kmemleak disabled oleg353-server: Loading modules from /home/green/git/lustre-release/lustre oleg353-server: detected 4 online CPUs by sysfs oleg353-server: MODOPTS_LIBCFS= oleg353-server: Force libcfs to create 2 CPU partitions oleg353-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg353-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg353-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg353-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg353-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg353-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg353-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg353-client.virtnet: -o user_xattr,flock 192.168.203.153@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 698 (time 1788756522.22 total 10.00 last 69.77) - open/close 1551 (time 1788756532.25 total 20.03 last 85.07) - open/close 2340 (time 1788756542.26 total 30.04 last 78.81) - open/close 3108 (time 1788756552.29 total 40.07 last 76.61) - open/close 3837 (time 1788756562.29 total 50.07 last 72.87) total: 4500 open/close in 58.18 seconds: 77.34 ops/second - /unlink 1463 (time 1788756584.77 total 10.00 last 146.27) - /unlink 3032 (time 1788756594.78 total 20.01 last 156.83) total: 4500 /unlink in 29.35 seconds: 153.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 821 (time 1788756618.52 total 10.00 last 82.07) - open/close 1516 (time 1788756628.54 total 20.03 last 69.34) - open/close 2378 (time 1788756638.55 total 30.04 last 86.13) - open/close 3028 (time 1788756648.56 total 40.04 last 64.96) - open/close 3950 (time 1788756658.57 total 50.05 last 92.10) total: 4500 open/close in 57.95 seconds: 77.65 ops/second - /unlink 1498 (time 1788756682.01 total 10.01 last 149.69) - /unlink 2765 (time 1788756692.01 total 20.01 last 126.69) - /unlink 4240 (time 1788756702.01 total 30.01 last 147.49) total: 4500 /unlink in 32.10 seconds: 140.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 865 (time 1788756718.75 total 10.01 last 86.43) - open/close 1651 (time 1788756728.76 total 20.01 last 78.56) - open/close 2344 (time 1788756738.77 total 30.02 last 69.22) - open/close 3180 (time 1788756748.78 total 40.04 last 83.49) - open/close 4001 (time 1788756758.79 total 50.05 last 82.03) total: 4500 open/close in 55.83 seconds: 80.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1779 (time 1788756780.40 total 10.00 last 177.85) - /unlink 3192 (time 1788756790.41 total 20.01 last 141.22) total: 4500 /unlink in 28.34 seconds: 158.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 849 (time 1788756813.09 total 10.02 last 84.76) - open/close 1609 (time 1788756823.10 total 20.03 last 75.92) - open/close 2410 (time 1788756833.10 total 30.03 last 80.09) - open/close 3148 (time 1788756843.13 total 40.06 last 73.55) - open/close 3739 (time 1788756853.14 total 50.07 last 59.04) - open/close 4432 (time 1788756863.15 total 60.08 last 69.21) total: 4500 open/close in 61.21 seconds: 73.52 ops/second - /unlink 1596 (time 1788756878.62 total 10.00 last 159.56) - /unlink 3023 (time 1788756888.62 total 20.00 last 142.68) total: 4500 /unlink in 29.45 seconds: 152.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 818 (time 1788756912.40 total 10.00 last 81.78) - open/close 1677 (time 1788756922.40 total 20.00 last 85.88) - open/close 2461 (time 1788756932.42 total 30.02 last 78.29) - open/close 3249 (time 1788756942.42 total 40.02 last 78.79) - open/close 4057 (time 1788756952.42 total 50.02 last 80.80) total: 4500 open/close in 55.43 seconds: 81.18 ops/second - /unlink 1495 (time 1788756971.52 total 10.00 last 149.45) - /unlink 3106 (time 1788756981.53 total 20.01 last 161.03) total: 4500 /unlink in 29.10 seconds: 154.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 799 (time 1788757005.61 total 10.01 last 79.80) - open/close 1568 (time 1788757015.62 total 20.02 last 76.81) - open/close 2249 (time 1788757025.64 total 30.04 last 67.96) - open/close 2834 (time 1788757035.64 total 40.05 last 58.47) - open/close 3411 (time 1788757045.64 total 50.05 last 57.69) - open/close 3929 (time 1788757055.66 total 60.06 last 51.74) - open/close 4440 (time 1788757065.66 total 70.07 last 51.08) total: 4500 open/close in 70.62 seconds: 63.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1463 (time 1788757080.87 total 10.00 last 146.26) - /unlink 2954 (time 1788757090.87 total 20.00 last 149.07) total: 4500 /unlink in 29.01 seconds: 155.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 842 (time 1788757114.12 total 10.00 last 84.18) - open/close 1689 (time 1788757124.13 total 20.01 last 84.62) - open/close 2531 (time 1788757134.15 total 30.03 last 84.09) - open/close 3365 (time 1788757144.15 total 40.03 last 83.38) - open/close 4093 (time 1788757154.17 total 50.05 last 72.66) total: 4500 open/close in 54.97 seconds: 81.86 ops/second - /unlink 1358 (time 1788757173.27 total 10.01 last 135.66) - /unlink 2859 (time 1788757183.27 total 20.01 last 150.05) - /unlink 4063 (time 1788757193.27 total 30.01 last 120.40) total: 4500 /unlink in 33.09 seconds: 135.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 707 (time 1788757211.48 total 10.02 last 70.59) - open/close 1260 (time 1788757221.49 total 20.02 last 55.25) - open/close 1932 (time 1788757231.50 total 30.04 last 67.10) - open/close 2798 (time 1788757241.51 total 40.05 last 86.55) - open/close 3579 (time 1788757251.52 total 50.05 last 78.05) - open/close 4415 (time 1788757261.52 total 60.05 last 83.60) total: 4500 open/close in 61.18 seconds: 73.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1713 (time 1788757276.49 total 10.01 last 171.20) - /unlink 3231 (time 1788757286.50 total 20.01 last 151.73) total: 4500 /unlink in 29.17 seconds: 154.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 791 (time 1788757311.10 total 10.01 last 79.06) - open/close 1575 (time 1788757321.11 total 20.01 last 78.36) - open/close 2379 (time 1788757331.12 total 30.02 last 80.34) - open/close 3053 (time 1788757341.13 total 40.03 last 67.33) - open/close 3680 (time 1788757351.13 total 50.04 last 62.66) - open/close 4304 (time 1788757361.16 total 60.06 last 62.26) total: 4500 open/close in 62.76 seconds: 71.70 ops/second - /unlink 1484 (time 1788757378.30 total 10.00 last 148.39) - /unlink 2932 (time 1788757388.31 total 20.01 last 144.72) total: 4500 /unlink in 29.79 seconds: 151.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 806 (time 1788757413.00 total 10.00 last 80.56) - open/close 1355 (time 1788757423.01 total 20.02 last 54.82) - open/close 1985 (time 1788757433.01 total 30.02 last 62.98) - open/close 2784 (time 1788757443.03 total 40.04 last 79.77) - open/close 3628 (time 1788757453.03 total 50.04 last 84.37) - open/close 4476 (time 1788757463.04 total 60.05 last 84.74) total: 4500 open/close in 60.33 seconds: 74.59 ops/second - /unlink 1626 (time 1788757477.81 total 10.00 last 162.57) - /unlink 3263 (time 1788757487.82 total 20.01 last 163.59) total: 4500 /unlink in 28.57 seconds: 157.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 888 (time 1788757511.10 total 10.02 last 88.66) - open/close 1758 (time 1788757521.10 total 20.02 last 86.94) - open/close 2664 (time 1788757531.11 total 30.03 last 90.56) - open/close 3597 (time 1788757541.11 total 40.03 last 93.28) - open/close 4486 (time 1788757551.12 total 50.03 last 88.87) total: 4500 open/close in 50.22 seconds: 89.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1572 (time 1788757565.80 total 10.00 last 157.16) - /unlink 3078 (time 1788757575.81 total 20.01 last 150.49) total: 4500 /unlink in 28.93 seconds: 155.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 762 (time 1788757599.76 total 10.01 last 76.15) - open/close 1577 (time 1788757609.77 total 20.01 last 81.44) - open/close 2291 (time 1788757619.77 total 30.02 last 71.37) - open/close 2866 (time 1788757629.78 total 40.03 last 57.42) - open/close 3446 (time 1788757639.79 total 50.04 last 57.97) - open/close 4179 (time 1788757649.79 total 60.04 last 73.28) total: 4500 open/close in 64.46 seconds: 69.81 ops/second - /unlink 1265 (time 1788757669.37 total 10.00 last 126.49) - /unlink 2519 (time 1788757679.37 total 20.00 last 125.39) - /unlink 3936 (time 1788757689.37 total 30.00 last 141.67) total: 4500 /unlink in 33.79 seconds: 133.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 830 (time 1788757707.64 total 10.00 last 82.98) - open/close 1587 (time 1788757717.64 total 20.00 last 75.69) - open/close 2463 (time 1788757727.65 total 30.01 last 87.53) - open/close 3226 (time 1788757737.66 total 40.02 last 76.25) - open/close 4045 (time 1788757747.67 total 50.03 last 81.82) total: 4500 open/close in 55.53 seconds: 81.04 ops/second - /unlink 1659 (time 1788757768.03 total 10.01 last 165.81) - /unlink 3321 (time 1788757778.03 total 20.01 last 166.18) total: 4500 /unlink in 28.03 seconds: 160.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 751 (time 1788757801.41 total 10.01 last 75.05) - open/close 1517 (time 1788757811.41 total 20.01 last 76.58) - open/close 2429 (time 1788757821.42 total 30.02 last 91.13) - open/close 3300 (time 1788757831.42 total 40.02 last 87.09) - open/close 4089 (time 1788757841.42 total 50.02 last 78.85) total: 4500 open/close in 55.74 seconds: 80.73 ops/second - /unlink 1573 (time 1788757861.01 total 10.00 last 157.28) - /unlink 3107 (time 1788757871.02 total 20.00 last 153.39) total: 4500 /unlink in 28.58 seconds: 157.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 849 (time 1788757894.40 total 10.00 last 84.88) - open/close 1769 (time 1788757904.41 total 20.01 last 91.90) - open/close 2651 (time 1788757914.43 total 30.03 last 88.10) - open/close 3543 (time 1788757924.43 total 40.03 last 89.14) - open/close 4330 (time 1788757934.44 total 50.04 last 78.61) total: 4500 open/close in 51.69 seconds: 87.05 ops/second - /unlink 1400 (time 1788757950.42 total 10.00 last 139.94) - /unlink 3023 (time 1788757960.42 total 20.01 last 162.25) total: 4500 /unlink in 29.29 seconds: 153.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 726 (time 1788757985.08 total 10.00 last 72.60) - open/close 1394 (time 1788757995.10 total 20.02 last 66.67) - open/close 2174 (time 1788758005.10 total 30.02 last 77.98) - open/close 2792 (time 1788758015.11 total 40.03 last 61.74) - open/close 3592 (time 1788758025.12 total 50.03 last 79.99) - open/close 4380 (time 1788758035.12 total 60.04 last 78.74) total: 4500 open/close in 61.51 seconds: 73.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1701 (time 1788758050.83 total 10.01 last 170.01) - /unlink 3356 (time 1788758060.83 total 20.01 last 165.46) total: 4500 /unlink in 27.51 seconds: 163.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 829 (time 1788758083.26 total 10.00 last 82.87) - open/close 1722 (time 1788758093.26 total 20.00 last 89.30) - open/close 2657 (time 1788758103.26 total 30.01 last 93.47) - open/close 3510 (time 1788758113.26 total 40.01 last 85.30) - open/close 4409 (time 1788758123.27 total 50.02 last 89.80) total: 4500 open/close in 51.10 seconds: 88.06 ops/second - /unlink 1582 (time 1788758138.29 total 10.00 last 158.19) - /unlink 3142 (time 1788758148.29 total 20.00 last 155.99) total: 4500 /unlink in 28.52 seconds: 157.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 796 (time 1788758171.38 total 10.01 last 79.55) - open/close 1635 (time 1788758181.39 total 20.02 last 83.83) - open/close 2279 (time 1788758191.40 total 30.02 last 64.35) - open/close 2949 (time 1788758201.42 total 40.04 last 66.88) - open/close 3813 (time 1788758211.43 total 50.06 last 86.26) total: 4500 open/close in 58.12 seconds: 77.43 ops/second - /unlink 1588 (time 1788758233.04 total 10.01 last 158.67) - /unlink 3299 (time 1788758243.05 total 20.01 last 171.05) total: 4500 /unlink in 27.88 seconds: 161.39 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11377: 95179 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg353-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg353-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg353-server stop mds service on oleg353-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg353-server stop mds service on oleg353-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg353-server unloading modules via unload_modules_local on: 'oleg353-server' oleg353-server: oleg353-server.virtnet: executing unload_modules_local oleg353-server: kmemleak disabled modules unloaded.