== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 05:50:48 (1789465848) Stopping clients: oleg451-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg451-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg451-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg451-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg451-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg451-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg451-server oleg451-server: oleg451-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg451-server' oleg451-server: oleg451-server.virtnet: executing load_modules_local oleg451-server: kmemleak disabled oleg451-server: Loading modules from /home/green/git/lustre-release/lustre oleg451-server: detected 4 online CPUs by sysfs oleg451-server: MODOPTS_LIBCFS= oleg451-server: Force libcfs to create 2 CPU partitions oleg451-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg451-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg451-server: oleg451-server.virtnet: executing set_default_debug -1 all pdsh@oleg451-client: oleg451-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg451-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg451-server: oleg451-server.virtnet: executing set_default_debug -1 all pdsh@oleg451-client: oleg451-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg451-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg451-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg451-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg451-server: oleg451-server.virtnet: executing set_default_debug -1 all pdsh@oleg451-client: oleg451-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg451-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg451-client.virtnet: -o user_xattr,flock 192.168.204.151@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 792 (time 1789466041.82 total 10.01 last 79.14) - open/close 1527 (time 1789466051.85 total 20.04 last 73.26) - open/close 2369 (time 1789466061.86 total 30.05 last 84.14) - open/close 3173 (time 1789466071.86 total 40.05 last 80.38) - open/close 3960 (time 1789466081.86 total 50.05 last 78.68) total: 4500 open/close in 56.80 seconds: 79.22 ops/second - /unlink 1503 (time 1789466102.92 total 10.00 last 150.24) - /unlink 2855 (time 1789466112.93 total 20.01 last 135.11) - /unlink 4343 (time 1789466122.93 total 30.01 last 148.79) total: 4500 /unlink in 31.30 seconds: 143.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 736 (time 1789466138.36 total 10.00 last 73.58) - open/close 1474 (time 1789466148.37 total 20.01 last 73.72) - open/close 2238 (time 1789466158.38 total 30.02 last 76.38) - open/close 2996 (time 1789466168.38 total 40.02 last 75.78) - open/close 3783 (time 1789466178.39 total 50.03 last 78.64) total: 4500 open/close in 58.96 seconds: 76.32 ops/second - /unlink 1446 (time 1789466202.41 total 10.01 last 144.40) - /unlink 2816 (time 1789466212.41 total 20.02 last 136.90) - /unlink 4227 (time 1789466222.42 total 30.03 last 140.94) total: 4500 /unlink in 31.92 seconds: 140.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 755 (time 1789466239.39 total 10.00 last 75.46) - open/close 1390 (time 1789466249.39 total 20.01 last 63.47) - open/close 2145 (time 1789466259.41 total 30.03 last 75.37) - open/close 2869 (time 1789466269.41 total 40.03 last 72.39) - open/close 3575 (time 1789466279.43 total 50.04 last 70.51) - open/close 4359 (time 1789466289.43 total 60.04 last 78.39) total: 4500 open/close in 61.78 seconds: 72.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1357 (time 1789466306.02 total 10.00 last 135.68) - /unlink 2635 (time 1789466316.02 total 20.00 last 127.77) - /unlink 3901 (time 1789466326.02 total 30.00 last 126.60) total: 4500 /unlink in 34.58 seconds: 130.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 820 (time 1789466345.62 total 10.00 last 81.97) - open/close 1547 (time 1789466355.62 total 20.01 last 72.68) - open/close 2299 (time 1789466365.63 total 30.01 last 75.16) - open/close 3076 (time 1789466375.64 total 40.02 last 77.63) - open/close 3808 (time 1789466385.64 total 50.03 last 73.15) total: 4500 open/close in 59.87 seconds: 75.16 ops/second - /unlink 1313 (time 1789466410.26 total 10.00 last 131.25) - /unlink 2826 (time 1789466420.26 total 20.01 last 151.26) - /unlink 4173 (time 1789466430.26 total 30.01 last 134.68) total: 4500 /unlink in 32.18 seconds: 139.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 733 (time 1789466447.90 total 10.01 last 73.19) - open/close 1528 (time 1789466457.91 total 20.02 last 79.47) - open/close 2232 (time 1789466467.92 total 30.03 last 70.36) - open/close 2828 (time 1789466477.92 total 40.03 last 59.56) - open/close 3511 (time 1789466487.93 total 50.04 last 68.27) - open/close 4314 (time 1789466497.93 total 60.04 last 80.29) total: 4500 open/close in 62.83 seconds: 71.62 ops/second - /unlink 1587 (time 1789466515.30 total 10.00 last 158.70) - /unlink 2929 (time 1789466525.30 total 20.01 last 134.08) - /unlink 4386 (time 1789466535.31 total 30.01 last 145.65) total: 4500 /unlink in 31.05 seconds: 144.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 810 (time 1789466551.35 total 10.01 last 80.96) - open/close 1580 (time 1789466561.36 total 20.01 last 76.94) - open/close 2282 (time 1789466571.36 total 30.01 last 70.19) - open/close 3044 (time 1789466581.37 total 40.02 last 76.12) - open/close 3837 (time 1789466591.38 total 50.03 last 79.29) total: 4500 open/close in 59.45 seconds: 75.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1053 (time 1789466615.84 total 10.00 last 105.25) - /unlink 2260 (time 1789466625.84 total 20.01 last 120.61) - /unlink 3661 (time 1789466635.84 total 30.01 last 140.10) total: 4500 /unlink in 36.41 seconds: 123.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 801 (time 1789466657.10 total 10.01 last 80.05) - open/close 1641 (time 1789466667.12 total 20.02 last 83.87) - open/close 2480 (time 1789466677.12 total 30.02 last 83.89) - open/close 3255 (time 1789466687.12 total 40.03 last 77.48) - open/close 4007 (time 1789466697.14 total 50.04 last 75.10) total: 4500 open/close in 56.63 seconds: 79.46 ops/second - /unlink 1349 (time 1789466718.18 total 10.00 last 134.88) - /unlink 2598 (time 1789466728.18 total 20.01 last 124.85) - /unlink 3989 (time 1789466738.18 total 30.01 last 139.07) total: 4500 /unlink in 33.50 seconds: 134.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 781 (time 1789466757.32 total 10.02 last 77.95) - open/close 1480 (time 1789466767.32 total 20.02 last 69.89) - open/close 2261 (time 1789466777.33 total 30.03 last 78.03) - open/close 3051 (time 1789466787.34 total 40.05 last 78.86) - open/close 3788 (time 1789466797.35 total 50.06 last 73.64) - open/close 4482 (time 1789466807.36 total 60.07 last 69.32) total: 4500 open/close in 60.39 seconds: 74.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1420 (time 1789466822.88 total 10.01 last 141.89) - /unlink 2857 (time 1789466832.88 total 20.01 last 143.61) - /unlink 4410 (time 1789466842.89 total 30.02 last 155.20) total: 4500 /unlink in 30.66 seconds: 146.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 775 (time 1789466858.19 total 10.01 last 77.46) - open/close 1536 (time 1789466868.20 total 20.01 last 76.07) - open/close 2234 (time 1789466878.21 total 30.03 last 69.68) - open/close 2945 (time 1789466888.22 total 40.03 last 71.07) - open/close 3579 (time 1789466898.23 total 50.04 last 63.32) - open/close 4216 (time 1789466908.24 total 60.05 last 63.64) total: 4500 open/close in 64.10 seconds: 70.21 ops/second - /unlink 1201 (time 1789466926.95 total 10.02 last 119.90) - /unlink 2614 (time 1789466936.96 total 20.02 last 141.24) - /unlink 4017 (time 1789466946.96 total 30.02 last 140.25) total: 4500 /unlink in 33.98 seconds: 132.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 736 (time 1789466965.40 total 10.00 last 73.59) - open/close 1446 (time 1789466975.41 total 20.01 last 70.94) - open/close 2005 (time 1789466985.43 total 30.03 last 55.79) - open/close 2763 (time 1789466995.44 total 40.04 last 75.72) - open/close 3514 (time 1789467005.44 total 50.04 last 75.08) - open/close 4262 (time 1789467015.44 total 60.04 last 74.79) total: 4500 open/close in 63.60 seconds: 70.76 ops/second - /unlink 1518 (time 1789467034.15 total 10.00 last 151.78) - /unlink 2870 (time 1789467044.15 total 20.00 last 135.19) - /unlink 4298 (time 1789467054.16 total 30.01 last 142.74) total: 4500 /unlink in 31.87 seconds: 141.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 747 (time 1789467071.50 total 10.01 last 74.62) - open/close 1458 (time 1789467081.50 total 20.01 last 71.07) - open/close 2140 (time 1789467091.50 total 30.02 last 68.19) - open/close 2956 (time 1789467101.51 total 40.02 last 81.55) - open/close 3692 (time 1789467111.52 total 50.04 last 73.50) - open/close 4428 (time 1789467121.52 total 60.04 last 73.59) total: 4500 open/close in 60.96 seconds: 73.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1554 (time 1789467137.41 total 10.01 last 155.30) - /unlink 2867 (time 1789467147.43 total 20.02 last 131.11) - /unlink 4278 (time 1789467157.43 total 30.02 last 141.07) total: 4500 /unlink in 31.80 seconds: 141.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 802 (time 1789467173.74 total 10.04 last 79.86) - open/close 1556 (time 1789467183.75 total 20.06 last 75.27) - open/close 2195 (time 1789467193.75 total 30.06 last 63.88) - open/close 2806 (time 1789467203.76 total 40.07 last 61.05) - open/close 3491 (time 1789467213.76 total 50.07 last 68.49) - open/close 4212 (time 1789467223.80 total 60.10 last 71.87) total: 4500 open/close in 63.47 seconds: 70.90 ops/second - /unlink 1440 (time 1789467241.57 total 10.00 last 143.97) - /unlink 2811 (time 1789467251.57 total 20.01 last 137.04) - /unlink 4306 (time 1789467261.57 total 30.01 last 149.43) total: 4500 /unlink in 31.46 seconds: 143.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 777 (time 1789467277.89 total 10.01 last 77.63) - open/close 1581 (time 1789467287.91 total 20.03 last 80.22) - open/close 2360 (time 1789467297.92 total 30.03 last 77.87) - open/close 3149 (time 1789467307.92 total 40.04 last 78.89) - open/close 3988 (time 1789467317.93 total 50.04 last 83.83) total: 4500 open/close in 56.34 seconds: 79.87 ops/second - /unlink 1381 (time 1789467338.14 total 10.00 last 138.09) - /unlink 2569 (time 1789467348.15 total 20.00 last 118.75) - /unlink 3879 (time 1789467358.15 total 30.01 last 130.99) total: 4500 /unlink in 34.28 seconds: 131.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 727 (time 1789467377.55 total 10.01 last 72.64) - open/close 1474 (time 1789467387.55 total 20.01 last 74.69) - open/close 2215 (time 1789467397.55 total 30.01 last 74.09) - open/close 2975 (time 1789467407.56 total 40.03 last 75.89) - open/close 3761 (time 1789467417.57 total 50.03 last 78.58) total: 4500 open/close in 59.00 seconds: 76.27 ops/second - /unlink 1419 (time 1789467440.78 total 10.01 last 141.76) - /unlink 2767 (time 1789467450.79 total 20.02 last 134.73) - /unlink 4208 (time 1789467460.79 total 30.02 last 144.09) total: 4500 /unlink in 32.44 seconds: 138.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 699 (time 1789467477.96 total 10.01 last 69.85) - open/close 1463 (time 1789467487.97 total 20.02 last 76.28) - open/close 2223 (time 1789467497.98 total 30.03 last 75.97) - open/close 2993 (time 1789467507.99 total 40.04 last 76.93) - open/close 3700 (time 1789467517.99 total 50.04 last 70.66) total: 4500 open/close in 59.91 seconds: 75.11 ops/second - /unlink 1470 (time 1789467541.42 total 10.00 last 146.99) - /unlink 2796 (time 1789467551.42 total 20.01 last 132.53) - /unlink 4304 (time 1789467561.42 total 30.01 last 150.80) total: 4500 /unlink in 31.60 seconds: 142.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 767 (time 1789467577.80 total 10.02 last 76.58) - open/close 1575 (time 1789467587.82 total 20.03 last 80.65) - open/close 2347 (time 1789467597.87 total 30.08 last 76.85) - open/close 3134 (time 1789467607.88 total 40.09 last 78.60) - open/close 3869 (time 1789467617.89 total 50.10 last 73.46) total: 4500 open/close in 59.42 seconds: 75.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1328 (time 1789467642.36 total 10.00 last 132.76) - /unlink 2651 (time 1789467652.37 total 20.02 last 132.13) - /unlink 4073 (time 1789467662.38 total 30.03 last 142.00) total: 4500 /unlink in 33.46 seconds: 134.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 768 (time 1789467681.57 total 10.00 last 76.78) - open/close 1556 (time 1789467691.58 total 20.01 last 78.72) - open/close 2306 (time 1789467701.58 total 30.01 last 75.00) - open/close 3177 (time 1789467711.58 total 40.02 last 87.04) - open/close 3990 (time 1789467721.59 total 50.03 last 81.23) total: 4500 open/close in 57.09 seconds: 78.83 ops/second - /unlink 1416 (time 1789467743.10 total 10.00 last 141.56) - /unlink 2844 (time 1789467753.10 total 20.01 last 142.72) - /unlink 4291 (time 1789467763.10 total 30.01 last 144.66) total: 4500 /unlink in 31.44 seconds: 143.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 777 (time 1789467779.54 total 10.02 last 77.56) - open/close 1585 (time 1789467789.54 total 20.02 last 80.76) - open/close 2368 (time 1789467799.54 total 30.03 last 78.27) - open/close 3104 (time 1789467809.55 total 40.03 last 73.54) - open/close 3852 (time 1789467819.56 total 50.04 last 74.76) total: 4500 open/close in 58.12 seconds: 77.42 ops/second - /unlink 1456 (time 1789467842.09 total 10.00 last 145.57) - /unlink 2849 (time 1789467852.10 total 20.01 last 139.23) - /unlink 4295 (time 1789467862.10 total 30.01 last 144.58) total: 4500 /unlink in 31.68 seconds: 142.02 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95092 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg451-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg451-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg451-server stop mds service on oleg451-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg451-server stop mds service on oleg451-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg451-server unloading modules via unload_modules_local on: 'oleg451-server' oleg451-server: oleg451-server.virtnet: executing unload_modules_local oleg451-server: kmemleak disabled modules unloaded.