== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 14:31:43 (1788805903) Stopping clients: oleg104-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg104-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg104-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg104-server oleg104-server: oleg104-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing load_modules_local oleg104-server: kmemleak disabled oleg104-server: Loading modules from /home/green/git/lustre-release/lustre oleg104-server: detected 4 online CPUs by sysfs oleg104-server: MODOPTS_LIBCFS= oleg104-server: Force libcfs to create 2 CPU partitions oleg104-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg104-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg104-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg104-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg104-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg104-client.virtnet: -o user_xattr,flock 192.168.201.104@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 993 (time 1788806091.76 total 10.00 last 99.25) - open/close 1970 (time 1788806101.76 total 20.01 last 97.69) - open/close 2923 (time 1788806111.76 total 30.01 last 95.28) - open/close 4041 (time 1788806121.76 total 40.01 last 111.74) total: 4500 open/close in 44.41 seconds: 101.33 ops/second - /unlink 2008 (time 1788806139.52 total 10.00 last 200.73) - /unlink 3903 (time 1788806149.53 total 20.01 last 189.43) total: 4500 /unlink in 23.32 seconds: 192.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 934 (time 1788806167.02 total 10.00 last 93.37) - open/close 2013 (time 1788806177.03 total 20.01 last 107.83) - open/close 3030 (time 1788806187.04 total 30.02 last 101.63) - open/close 4017 (time 1788806197.04 total 40.02 last 98.69) total: 4500 open/close in 44.25 seconds: 101.70 ops/second - /unlink 1932 (time 1788806214.55 total 10.01 last 193.09) - /unlink 3919 (time 1788806224.56 total 20.01 last 198.57) total: 4500 /unlink in 22.95 seconds: 196.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1045 (time 1788806240.76 total 10.01 last 104.44) - open/close 1979 (time 1788806250.76 total 20.01 last 93.39) - open/close 3119 (time 1788806260.76 total 30.01 last 113.97) - open/close 4168 (time 1788806270.79 total 40.04 last 104.61) total: 4500 open/close in 43.75 seconds: 102.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1667 (time 1788806287.75 total 10.02 last 166.43) - /unlink 3089 (time 1788806297.75 total 20.02 last 142.20) total: 4500 /unlink in 27.68 seconds: 162.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1018 (time 1788806318.56 total 10.01 last 101.71) - open/close 1999 (time 1788806328.56 total 20.01 last 98.09) - open/close 3080 (time 1788806338.57 total 30.03 last 107.93) - open/close 4108 (time 1788806348.58 total 40.03 last 102.76) total: 4500 open/close in 43.60 seconds: 103.21 ops/second - /unlink 1902 (time 1788806365.33 total 10.00 last 190.11) - /unlink 4144 (time 1788806375.33 total 20.01 last 224.17) total: 4500 /unlink in 21.74 seconds: 207.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1030 (time 1788806390.28 total 10.00 last 102.98) - open/close 2007 (time 1788806400.28 total 20.01 last 97.63) - open/close 2978 (time 1788806410.30 total 30.02 last 96.97) - open/close 3965 (time 1788806420.30 total 40.03 last 98.66) total: 4500 open/close in 45.15 seconds: 99.67 ops/second - /unlink 2165 (time 1788806438.74 total 10.00 last 216.43) - /unlink 4012 (time 1788806448.74 total 20.00 last 184.70) total: 4500 /unlink in 23.14 seconds: 194.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1062 (time 1788806465.75 total 10.00 last 106.18) - open/close 2066 (time 1788806475.76 total 20.01 last 100.32) - open/close 3127 (time 1788806485.76 total 30.01 last 106.08) - open/close 4241 (time 1788806495.77 total 40.01 last 111.38) total: 4500 open/close in 42.27 seconds: 106.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1940 (time 1788806510.77 total 10.00 last 193.95) - /unlink 3834 (time 1788806520.77 total 20.00 last 189.36) total: 4500 /unlink in 23.55 seconds: 191.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1105 (time 1788806537.52 total 10.00 last 110.45) - open/close 2197 (time 1788806547.53 total 20.01 last 109.14) - open/close 3245 (time 1788806557.54 total 30.02 last 104.72) - open/close 4320 (time 1788806567.55 total 40.03 last 107.37) total: 4500 open/close in 41.60 seconds: 108.18 ops/second - /unlink 1982 (time 1788806582.82 total 10.00 last 198.19) - /unlink 4139 (time 1788806592.82 total 20.01 last 215.55) total: 4500 /unlink in 21.41 seconds: 210.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1142 (time 1788806607.67 total 10.01 last 114.13) - open/close 2234 (time 1788806617.69 total 20.03 last 108.93) - open/close 3302 (time 1788806627.70 total 30.04 last 106.75) - open/close 4267 (time 1788806637.71 total 40.05 last 96.41) total: 4500 open/close in 42.39 seconds: 106.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1869 (time 1788806653.27 total 10.00 last 186.84) - /unlink 4258 (time 1788806663.28 total 20.01 last 238.82) total: 4500 /unlink in 21.28 seconds: 211.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 967 (time 1788806677.64 total 10.00 last 96.69) - open/close 1998 (time 1788806687.65 total 20.01 last 103.03) - open/close 3052 (time 1788806697.65 total 30.01 last 105.33) - open/close 4160 (time 1788806707.66 total 40.02 last 110.75) total: 4500 open/close in 43.31 seconds: 103.90 ops/second - /unlink 2553 (time 1788806723.52 total 10.01 last 255.06) total: 4500 /unlink in 17.50 seconds: 257.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1212 (time 1788806744.06 total 10.01 last 121.08) - open/close 2386 (time 1788806754.07 total 20.01 last 117.36) - open/close 3612 (time 1788806764.07 total 30.02 last 122.57) total: 4500 open/close in 37.14 seconds: 121.17 ops/second - /unlink 2190 (time 1788806783.86 total 10.01 last 218.84) - /unlink 4290 (time 1788806793.86 total 20.01 last 209.99) total: 4500 /unlink in 20.87 seconds: 215.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1051 (time 1788806807.73 total 10.00 last 105.08) - open/close 2124 (time 1788806817.74 total 20.01 last 107.24) - open/close 3311 (time 1788806827.74 total 30.01 last 118.70) total: 4500 open/close in 39.14 seconds: 114.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 3126 (time 1788806849.03 total 10.00 last 312.56) total: 4500 /unlink in 13.71 seconds: 328.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1348 (time 1788806864.65 total 10.00 last 134.78) - open/close 2718 (time 1788806874.65 total 20.00 last 136.99) - open/close 4008 (time 1788806884.65 total 30.01 last 128.91) total: 4500 open/close in 33.69 seconds: 133.58 ops/second - /unlink 2450 (time 1788806900.49 total 10.00 last 244.88) total: 4500 /unlink in 18.22 seconds: 247.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1432 (time 1788806921.35 total 10.00 last 143.16) - open/close 2604 (time 1788806931.35 total 20.00 last 117.19) - open/close 3750 (time 1788806941.35 total 30.01 last 114.55) total: 4500 open/close in 36.00 seconds: 124.99 ops/second - /unlink 2490 (time 1788806959.78 total 10.00 last 249.00) total: 4500 /unlink in 18.90 seconds: 238.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1226 (time 1788806981.30 total 10.01 last 122.45) - open/close 2545 (time 1788806991.31 total 20.02 last 131.74) - open/close 3767 (time 1788807001.31 total 30.03 last 122.15) total: 4500 open/close in 35.76 seconds: 125.83 ops/second - /unlink 2164 (time 1788807019.60 total 10.00 last 216.40) - /unlink 4262 (time 1788807029.60 total 20.00 last 209.70) total: 4500 /unlink in 21.14 seconds: 212.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1121 (time 1788807043.36 total 10.01 last 112.03) - open/close 2218 (time 1788807053.36 total 20.01 last 109.68) - open/close 3622 (time 1788807063.36 total 30.01 last 140.37) total: 4500 open/close in 36.82 seconds: 122.23 ops/second - /unlink 2960 (time 1788807082.36 total 10.00 last 295.89) total: 4500 /unlink in 15.19 seconds: 296.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1470 (time 1788807099.70 total 10.00 last 146.99) - open/close 2933 (time 1788807109.70 total 20.00 last 146.24) - open/close 4460 (time 1788807119.70 total 30.01 last 152.64) total: 4500 open/close in 30.26 seconds: 148.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3877 (time 1788807131.68 total 10.00 last 387.70) total: 4500 /unlink in 11.86 seconds: 379.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1341 (time 1788807145.32 total 10.00 last 134.05) - open/close 2066 (time 1788807155.32 total 20.01 last 72.46) - open/close 2791 (time 1788807165.33 total 30.01 last 72.48) - open/close 3587 (time 1788807175.33 total 40.02 last 79.54) - open/close 4183 (time 1788807185.35 total 50.03 last 59.53) total: 4500 open/close in 54.67 seconds: 82.32 ops/second - /unlink 1703 (time 1788807205.51 total 10.00 last 170.30) - /unlink 3611 (time 1788807215.51 total 20.00 last 190.75) total: 4500 /unlink in 25.13 seconds: 179.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 660 (time 1788807234.11 total 10.01 last 65.92) - open/close 1299 (time 1788807244.14 total 20.04 last 63.75) - open/close 1875 (time 1788807254.16 total 30.05 last 57.49) - open/close 2705 (time 1788807264.16 total 40.06 last 82.97) - open/close 3688 (time 1788807274.16 total 50.06 last 98.30) total: 4500 open/close in 57.99 seconds: 77.59 ops/second - /unlink 1862 (time 1788807295.72 total 10.00 last 186.15) - /unlink 3461 (time 1788807305.73 total 20.01 last 159.86) total: 4500 /unlink in 25.22 seconds: 178.45 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95152 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg104-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server unloading modules via unload_modules_local on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing unload_modules_local oleg104-server: kmemleak disabled modules unloaded.