== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:19:37 (1787444377) Stopping clients: oleg246-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg246-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg246-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg246-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg246-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg246-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg246-server oleg246-server: oleg246-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg246-server' oleg246-server: oleg246-server.virtnet: executing load_modules_local oleg246-server: Loading modules from /home/green/git/lustre-release/lustre oleg246-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg246-server: detected 4 online CPUs by sysfs oleg246-server: MODOPTS_LIBCFS= oleg246-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg246-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg246-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg246-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg246-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg246-server: oleg246-server.virtnet: executing set_default_debug -1 all pdsh@oleg246-client: oleg246-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg246-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg246-client.virtnet: -o user_xattr,flock 192.168.202.146@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 800 (time 1787444591.31 total 10.00 last 79.98) - open/close 1586 (time 1787444601.32 total 20.01 last 78.54) - open/close 2417 (time 1787444611.37 total 30.06 last 82.69) - open/close 3254 (time 1787444621.38 total 40.07 last 83.65) - open/close 3961 (time 1787444631.40 total 50.09 last 70.54) total: 4500 open/close in 57.11 seconds: 78.80 ops/second - /unlink 1718 (time 1787444653.21 total 10.00 last 171.76) - /unlink 3472 (time 1787444663.22 total 20.01 last 175.35) total: 4500 /unlink in 26.51 seconds: 169.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 971 (time 1787444683.18 total 10.02 last 96.86) - open/close 1849 (time 1787444693.19 total 20.03 last 87.73) - open/close 2712 (time 1787444703.19 total 30.04 last 86.27) - open/close 3600 (time 1787444713.20 total 40.05 last 88.72) - open/close 4429 (time 1787444723.20 total 50.05 last 82.85) total: 4500 open/close in 50.81 seconds: 88.57 ops/second - /unlink 1329 (time 1787444738.31 total 10.00 last 132.88) - /unlink 2807 (time 1787444748.31 total 20.01 last 147.75) - /unlink 4184 (time 1787444758.31 total 30.01 last 137.63) total: 4500 /unlink in 32.34 seconds: 139.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 847 (time 1787444775.67 total 10.02 last 84.54) - open/close 1627 (time 1787444785.67 total 20.03 last 77.95) - open/close 2423 (time 1787444795.68 total 30.03 last 79.57) - open/close 3250 (time 1787444805.68 total 40.04 last 82.65) - open/close 4020 (time 1787444815.69 total 50.04 last 76.95) total: 4500 open/close in 56.11 seconds: 80.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1529 (time 1787444835.83 total 10.00 last 152.85) - /unlink 3091 (time 1787444845.83 total 20.00 last 156.17) total: 4500 /unlink in 28.96 seconds: 155.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 741 (time 1787444870.05 total 10.01 last 74.05) - open/close 1534 (time 1787444880.06 total 20.01 last 79.26) - open/close 2185 (time 1787444890.06 total 30.01 last 65.08) - open/close 2976 (time 1787444900.07 total 40.02 last 79.03) - open/close 3790 (time 1787444910.08 total 50.03 last 81.33) total: 4500 open/close in 59.57 seconds: 75.54 ops/second - /unlink 1478 (time 1787444934.68 total 10.00 last 147.73) - /unlink 2546 (time 1787444944.68 total 20.01 last 106.80) - /unlink 3943 (time 1787444954.68 total 30.01 last 139.68) total: 4500 /unlink in 34.76 seconds: 129.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 895 (time 1787444974.45 total 10.00 last 89.46) - open/close 1774 (time 1787444984.46 total 20.02 last 87.81) - open/close 2566 (time 1787444994.46 total 30.02 last 79.19) - open/close 3564 (time 1787445004.46 total 40.02 last 99.76) - open/close 4408 (time 1787445014.47 total 50.03 last 84.36) total: 4500 open/close in 51.06 seconds: 88.13 ops/second - /unlink 1699 (time 1787445029.53 total 10.00 last 169.88) - /unlink 3240 (time 1787445039.54 total 20.01 last 154.03) total: 4500 /unlink in 28.28 seconds: 159.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 596 (time 1787445062.49 total 10.01 last 59.53) - open/close 1271 (time 1787445072.49 total 20.02 last 67.46) - open/close 2061 (time 1787445082.52 total 30.04 last 78.84) - open/close 2962 (time 1787445092.52 total 40.04 last 90.07) - open/close 3913 (time 1787445102.52 total 50.05 last 95.05) total: 4500 open/close in 56.77 seconds: 79.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1705 (time 1787445122.88 total 10.01 last 170.32) - /unlink 2973 (time 1787445132.88 total 20.01 last 126.80) - /unlink 3988 (time 1787445142.89 total 30.02 last 101.45) total: 4500 /unlink in 33.41 seconds: 134.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 943 (time 1787445161.32 total 10.00 last 94.28) - open/close 1750 (time 1787445171.32 total 20.00 last 80.69) - open/close 2577 (time 1787445181.33 total 30.01 last 82.65) - open/close 3390 (time 1787445191.33 total 40.01 last 81.28) - open/close 4106 (time 1787445201.34 total 50.02 last 71.52) total: 4500 open/close in 54.93 seconds: 81.92 ops/second - /unlink 1553 (time 1787445221.14 total 10.00 last 155.25) - /unlink 3021 (time 1787445231.14 total 20.01 last 146.73) total: 4500 /unlink in 29.83 seconds: 150.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 716 (time 1787445256.34 total 10.01 last 71.52) - open/close 1512 (time 1787445266.34 total 20.01 last 79.59) - open/close 2253 (time 1787445276.35 total 30.02 last 74.06) - open/close 3014 (time 1787445286.35 total 40.03 last 76.04) - open/close 3757 (time 1787445296.36 total 50.03 last 74.28) total: 4500 open/close in 59.39 seconds: 75.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1321 (time 1787445320.66 total 10.00 last 132.07) - /unlink 2847 (time 1787445330.66 total 20.00 last 152.60) - /unlink 4378 (time 1787445340.67 total 30.01 last 153.04) total: 4500 /unlink in 30.95 seconds: 145.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 817 (time 1787445356.11 total 10.01 last 81.65) - open/close 1545 (time 1787445366.13 total 20.02 last 72.71) - open/close 2360 (time 1787445376.13 total 30.03 last 81.44) - open/close 3272 (time 1787445386.14 total 40.03 last 91.13) - open/close 4081 (time 1787445396.15 total 50.05 last 80.81) total: 4500 open/close in 55.37 seconds: 81.28 ops/second - /unlink 1532 (time 1787445415.74 total 10.02 last 152.93) - /unlink 3064 (time 1787445425.74 total 20.02 last 153.11) total: 4500 /unlink in 29.95 seconds: 150.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 870 (time 1787445450.70 total 10.00 last 86.99) - open/close 1511 (time 1787445460.71 total 20.01 last 64.04) - open/close 2244 (time 1787445470.72 total 30.02 last 73.22) - open/close 2878 (time 1787445480.73 total 40.03 last 63.37) - open/close 3492 (time 1787445490.73 total 50.03 last 61.36) - open/close 4179 (time 1787445500.74 total 60.04 last 68.64) total: 4500 open/close in 65.17 seconds: 69.05 ops/second - /unlink 1495 (time 1787445521.03 total 10.02 last 149.23) - /unlink 2683 (time 1787445531.03 total 20.02 last 118.75) - /unlink 3895 (time 1787445541.03 total 30.02 last 121.18) total: 4500 /unlink in 34.34 seconds: 131.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 649 (time 1787445560.80 total 10.03 last 64.71) - open/close 1252 (time 1787445570.81 total 20.04 last 60.25) - open/close 1869 (time 1787445580.83 total 30.05 last 61.61) - open/close 2380 (time 1787445590.83 total 40.06 last 51.08) - open/close 2829 (time 1787445600.83 total 50.06 last 44.88) - open/close 3437 (time 1787445610.84 total 60.06 last 60.80) - open/close 4125 (time 1787445620.84 total 70.07 last 68.77) total: 4500 open/close in 73.66 seconds: 61.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1689 (time 1787445638.45 total 10.01 last 168.80) - /unlink 3187 (time 1787445648.45 total 20.01 last 149.73) - /unlink 4403 (time 1787445658.46 total 30.02 last 121.54) total: 4500 /unlink in 30.99 seconds: 145.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 890 (time 1787445675.19 total 10.01 last 88.93) - open/close 1730 (time 1787445685.20 total 20.02 last 83.92) - open/close 2570 (time 1787445695.21 total 30.02 last 83.98) - open/close 3403 (time 1787445705.21 total 40.02 last 83.26) - open/close 4201 (time 1787445715.22 total 50.03 last 79.75) total: 4500 open/close in 53.93 seconds: 83.44 ops/second - /unlink 1406 (time 1787445734.35 total 10.00 last 140.54) - /unlink 2910 (time 1787445744.35 total 20.01 last 150.38) - /unlink 4361 (time 1787445754.35 total 30.01 last 145.06) total: 4500 /unlink in 31.52 seconds: 142.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 836 (time 1787445771.63 total 10.01 last 83.49) - open/close 1658 (time 1787445781.64 total 20.02 last 82.16) - open/close 2417 (time 1787445791.65 total 30.03 last 75.81) - open/close 3146 (time 1787445801.66 total 40.04 last 72.80) - open/close 3883 (time 1787445811.66 total 50.05 last 73.68) total: 4500 open/close in 59.15 seconds: 76.07 ops/second - /unlink 1545 (time 1787445836.23 total 10.00 last 154.44) - /unlink 2859 (time 1787445846.24 total 20.01 last 131.31) - /unlink 4211 (time 1787445856.25 total 30.02 last 135.05) total: 4500 /unlink in 32.51 seconds: 138.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 857 (time 1787445873.95 total 10.00 last 85.69) - open/close 1553 (time 1787445883.95 total 20.01 last 69.56) - open/close 2406 (time 1787445893.95 total 30.01 last 85.30) - open/close 3127 (time 1787445903.96 total 40.01 last 72.08) - open/close 3811 (time 1787445913.97 total 50.03 last 68.30) - open/close 4416 (time 1787445923.99 total 60.05 last 60.36) total: 4500 open/close in 61.28 seconds: 73.44 ops/second - /unlink 1533 (time 1787445940.32 total 10.00 last 153.29) - /unlink 2765 (time 1787445950.33 total 20.01 last 123.12) - /unlink 4047 (time 1787445960.33 total 30.01 last 128.19) total: 4500 /unlink in 33.51 seconds: 134.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 663 (time 1787445980.16 total 10.00 last 66.27) - open/close 1355 (time 1787445990.16 total 20.01 last 69.18) - open/close 2135 (time 1787446000.16 total 30.01 last 77.95) - open/close 3111 (time 1787446010.17 total 40.01 last 97.59) - open/close 3966 (time 1787446020.17 total 50.02 last 85.43) total: 4500 open/close in 55.63 seconds: 80.89 ops/second - /unlink 1563 (time 1787446040.29 total 10.00 last 156.25) - /unlink 2989 (time 1787446050.30 total 20.01 last 142.52) total: 4500 /unlink in 28.98 seconds: 155.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 939 (time 1787446074.32 total 10.00 last 93.89) - open/close 1872 (time 1787446084.37 total 20.05 last 92.81) - open/close 2606 (time 1787446094.39 total 30.06 last 73.32) - open/close 3084 (time 1787446104.39 total 40.07 last 47.77) - open/close 3807 (time 1787446114.39 total 50.07 last 72.29) total: 4500 open/close in 58.00 seconds: 77.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1153 (time 1787446137.08 total 10.00 last 115.29) - /unlink 2496 (time 1787446147.09 total 20.00 last 134.25) - /unlink 4021 (time 1787446157.09 total 30.01 last 152.45) total: 4500 /unlink in 33.26 seconds: 135.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 826 (time 1787446175.76 total 10.00 last 82.59) - open/close 1594 (time 1787446185.76 total 20.01 last 76.75) - open/close 2319 (time 1787446195.77 total 30.02 last 72.43) - open/close 3096 (time 1787446205.78 total 40.02 last 77.65) - open/close 3967 (time 1787446215.79 total 50.03 last 87.01) total: 4500 open/close in 57.44 seconds: 78.34 ops/second - /unlink 1488 (time 1787446237.63 total 10.01 last 148.70) - /unlink 2855 (time 1787446247.63 total 20.01 last 136.70) - /unlink 4107 (time 1787446257.63 total 30.01 last 125.14) total: 4500 /unlink in 33.52 seconds: 134.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 647 (time 1787446277.82 total 10.00 last 64.69) - open/close 1288 (time 1787446287.83 total 20.01 last 64.03) - open/close 2061 (time 1787446297.87 total 30.05 last 77.01) - open/close 2712 (time 1787446307.88 total 40.06 last 65.07) - open/close 3535 (time 1787446317.89 total 50.07 last 82.22) - open/close 4333 (time 1787446327.90 total 60.08 last 79.68) total: 4500 open/close in 62.68 seconds: 71.79 ops/second - /unlink 1522 (time 1787446346.70 total 10.00 last 152.18) - /unlink 3263 (time 1787446356.70 total 20.01 last 174.02) total: 4500 /unlink in 27.66 seconds: 162.72 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95340 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg246-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg246-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg246-server stop mds service on oleg246-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg246-server stop mds service on oleg246-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg246-server unloading modules via unload_modules_local on: 'oleg246-server' oleg246-server: oleg246-server.virtnet: executing unload_modules_local oleg246-server: modules unloaded.