== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 17:56:16 (1781819776) Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg335-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg335-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg335-server oleg335-server: oleg335-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1045: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing load_modules_local oleg335-server: Loading modules from /home/green/git/lustre-release/lustre oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1045: echo: write error: Device or resource busy oleg335-server: detected 4 online CPUs by sysfs oleg335-server: MODOPTS_LIBCFS= oleg335-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg335-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg335-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg335-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg335-client.virtnet: -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 2865 (time 1781819873.36 total 10.00 last 286.43) total: 4500 open/close in 18.65 seconds: 241.23 ops/second total: 4500 /unlink in 9.23 seconds: 487.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 2715 (time 1781819904.48 total 10.00 last 271.50) total: 4500 open/close in 16.43 seconds: 273.86 ops/second - /unlink 3783 (time 1781819922.58 total 10.01 last 377.99) total: 4500 /unlink in 13.46 seconds: 334.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1849 (time 1781819938.43 total 10.00 last 184.83) - open/close 3484 (time 1781819948.44 total 20.01 last 163.43) total: 4500 open/close in 25.40 seconds: 177.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 3319 (time 1781819965.54 total 10.00 last 331.81) total: 4500 /unlink in 15.38 seconds: 292.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1408 (time 1781819982.90 total 10.01 last 140.71) - open/close 3090 (time 1781819992.90 total 20.01 last 168.11) - open/close 4460 (time 1781820002.91 total 30.02 last 136.90) total: 4500 open/close in 30.27 seconds: 148.68 ops/second total: 4500 /unlink in 8.69 seconds: 518.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1751 (time 1781820025.16 total 10.00 last 175.03) - open/close 3151 (time 1781820035.17 total 20.01 last 139.93) - open/close 4417 (time 1781820045.17 total 30.01 last 126.55) total: 4500 open/close in 30.47 seconds: 147.67 ops/second - /unlink 3347 (time 1781820057.36 total 10.00 last 334.60) total: 4500 /unlink in 14.81 seconds: 303.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1581 (time 1781820074.09 total 10.01 last 157.94) - open/close 3343 (time 1781820084.09 total 20.01 last 176.13) total: 4500 open/close in 26.19 seconds: 171.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 4361 (time 1781820102.14 total 10.00 last 436.10) total: 4500 /unlink in 10.44 seconds: 431.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1517 (time 1781820114.47 total 10.00 last 151.70) - open/close 2674 (time 1781820124.47 total 20.00 last 115.65) - open/close 3763 (time 1781820134.47 total 30.01 last 108.87) total: 4500 open/close in 34.60 seconds: 130.05 ops/second total: 4500 /unlink in 8.53 seconds: 527.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 2131 (time 1781820160.81 total 10.00 last 213.08) - open/close 3868 (time 1781820170.81 total 20.01 last 173.54) total: 4500 open/close in 26.94 seconds: 167.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1651 (time 1781820190.41 total 10.00 last 165.02) - /unlink 3555 (time 1781820200.41 total 20.01 last 190.33) total: 4500 /unlink in 24.09 seconds: 186.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1886 (time 1781820216.51 total 10.00 last 188.54) - open/close 3451 (time 1781820226.51 total 20.01 last 156.45) total: 4500 open/close in 26.34 seconds: 170.87 ops/second - /unlink 2940 (time 1781820244.81 total 10.00 last 293.86) total: 4500 /unlink in 16.57 seconds: 271.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1776 (time 1781820263.43 total 10.00 last 177.54) - open/close 3340 (time 1781820273.43 total 20.01 last 156.35) total: 4500 open/close in 26.05 seconds: 172.77 ops/second - /unlink 3081 (time 1781820291.34 total 10.00 last 308.01) total: 4500 /unlink in 15.47 seconds: 290.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1262 (time 1781820308.94 total 10.00 last 126.14) - open/close 2822 (time 1781820318.94 total 20.01 last 155.94) total: 4500 open/close in 28.34 seconds: 158.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2898 (time 1781820339.14 total 10.00 last 289.74) total: 4500 /unlink in 14.73 seconds: 305.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1270 (time 1781820355.65 total 10.01 last 126.92) - open/close 2338 (time 1781820365.66 total 20.01 last 106.72) - open/close 3342 (time 1781820375.66 total 30.02 last 100.38) - open/close 4271 (time 1781820385.66 total 40.02 last 92.85) total: 4500 open/close in 42.56 seconds: 105.74 ops/second - /unlink 2192 (time 1781820400.86 total 10.00 last 219.14) - /unlink 4220 (time 1781820410.87 total 20.01 last 202.70) total: 4500 /unlink in 21.56 seconds: 208.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1042 (time 1781820425.17 total 10.01 last 104.13) - open/close 1943 (time 1781820435.17 total 20.01 last 90.10) - open/close 2934 (time 1781820445.17 total 30.01 last 99.05) - open/close 3976 (time 1781820455.18 total 40.02 last 104.17) total: 4500 open/close in 44.41 seconds: 101.32 ops/second - /unlink 2457 (time 1781820471.75 total 10.00 last 245.59) total: 4500 /unlink in 18.67 seconds: 240.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1776 (time 1781820493.04 total 10.00 last 177.54) - open/close 3007 (time 1781820503.05 total 20.01 last 123.07) - open/close 4403 (time 1781820513.05 total 30.01 last 139.56) total: 4500 open/close in 30.73 seconds: 146.45 ops/second - /unlink 3052 (time 1781820525.81 total 10.00 last 305.19) total: 4500 /unlink in 14.78 seconds: 304.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1142 (time 1781820542.71 total 10.00 last 114.15) - open/close 2176 (time 1781820552.72 total 20.00 last 103.39) - open/close 3256 (time 1781820562.72 total 30.01 last 107.96) - open/close 4427 (time 1781820572.73 total 40.02 last 116.97) total: 4500 open/close in 40.68 seconds: 110.62 ops/second - /unlink 3433 (time 1781820585.64 total 10.00 last 343.30) total: 4500 /unlink in 13.30 seconds: 338.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1311 (time 1781820600.74 total 10.00 last 131.04) - open/close 2500 (time 1781820610.75 total 20.01 last 118.79) - open/close 3823 (time 1781820620.75 total 30.02 last 132.25) total: 4500 open/close in 35.73 seconds: 125.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1783 (time 1781820639.44 total 10.00 last 178.26) - /unlink 4199 (time 1781820649.44 total 20.00 last 241.59) total: 4500 /unlink in 21.00 seconds: 214.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1089 (time 1781820662.58 total 10.01 last 108.83) - open/close 2297 (time 1781820672.58 total 20.01 last 120.77) - open/close 3394 (time 1781820682.58 total 30.01 last 109.69) - open/close 4453 (time 1781820692.59 total 40.01 last 105.84) total: 4500 open/close in 40.46 seconds: 111.21 ops/second - /unlink 1795 (time 1781820705.30 total 10.00 last 179.49) - /unlink 3452 (time 1781820715.31 total 20.00 last 165.63) total: 4500 /unlink in 24.93 seconds: 180.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1093 (time 1781820732.61 total 10.00 last 109.29) - open/close 2032 (time 1781820742.62 total 20.00 last 93.88) - open/close 2935 (time 1781820752.62 total 30.01 last 90.27) - open/close 4052 (time 1781820762.62 total 40.01 last 111.68) total: 4500 open/close in 43.49 seconds: 103.48 ops/second - /unlink 2680 (time 1781820778.15 total 10.01 last 267.85) total: 4500 /unlink in 17.21 seconds: 261.44 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11225: 94217 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg335-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg335-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server stop mds service on oleg335-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server stop mds service on oleg335-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg335-server unloading modules via unload_modules_local on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing unload_modules_local oleg335-server: modules unloaded.