== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 08:01:43 (1781611303) Stopping clients: oleg352-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg352-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg352-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg352-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg352-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg352-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg352-server oleg352-server: oleg352-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1045: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg352-server' oleg352-server: oleg352-server.virtnet: executing load_modules_local oleg352-server: Loading modules from /home/green/git/lustre-release/lustre oleg352-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1045: echo: write error: Device or resource busy oleg352-server: detected 4 online CPUs by sysfs oleg352-server: MODOPTS_LIBCFS= oleg352-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg352-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug -1 all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg352-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg352-server: oleg352-server.virtnet: executing set_default_debug -1 all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg352-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg352-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg352-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg352-server: oleg352-server.virtnet: executing set_default_debug -1 all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1140 (time 1781611446.60 total 10.01 last 113.93) - open/close 2342 (time 1781611456.61 total 20.01 last 120.18) - open/close 3803 (time 1781611466.61 total 30.01 last 146.05) total: 4500 open/close in 35.78 seconds: 125.79 ops/second - /unlink 2176 (time 1781611484.77 total 10.00 last 217.52) - /unlink 4337 (time 1781611494.77 total 20.01 last 216.01) total: 4500 /unlink in 20.78 seconds: 216.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1205 (time 1781611508.23 total 10.00 last 120.48) - open/close 2364 (time 1781611518.23 total 20.00 last 115.88) - open/close 3508 (time 1781611528.23 total 30.00 last 114.39) total: 4500 open/close in 38.72 seconds: 116.21 ops/second - /unlink 2218 (time 1781611549.37 total 10.00 last 221.80) total: 4500 /unlink in 19.55 seconds: 230.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1076 (time 1781611571.36 total 10.00 last 107.55) - open/close 2066 (time 1781611581.37 total 20.02 last 98.86) - open/close 3156 (time 1781611591.37 total 30.02 last 108.96) - open/close 4225 (time 1781611601.38 total 40.02 last 106.88) total: 4500 open/close in 42.64 seconds: 105.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2292 (time 1781611616.24 total 10.00 last 229.15) total: 4500 /unlink in 18.93 seconds: 237.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1409 (time 1781611637.24 total 10.00 last 140.89) - open/close 2743 (time 1781611647.24 total 20.00 last 133.38) - open/close 3910 (time 1781611657.24 total 30.00 last 116.67) total: 4500 open/close in 35.00 seconds: 128.59 ops/second - /unlink 2074 (time 1781611674.68 total 10.00 last 207.36) - /unlink 4197 (time 1781611684.68 total 20.00 last 212.26) total: 4500 /unlink in 21.28 seconds: 211.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1121 (time 1781611698.28 total 10.01 last 111.98) - open/close 2166 (time 1781611708.28 total 20.02 last 104.43) - open/close 3290 (time 1781611718.29 total 30.03 last 112.27) - open/close 4325 (time 1781611728.30 total 40.04 last 103.43) total: 4500 open/close in 41.84 seconds: 107.54 ops/second - /unlink 1931 (time 1781611742.71 total 10.00 last 193.09) - /unlink 4106 (time 1781611752.71 total 20.00 last 217.45) total: 4500 /unlink in 21.91 seconds: 205.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1032 (time 1781611767.22 total 10.00 last 103.16) - open/close 2066 (time 1781611777.22 total 20.00 last 103.39) - open/close 3150 (time 1781611787.22 total 30.01 last 108.40) - open/close 4183 (time 1781611797.23 total 40.01 last 103.24) total: 4500 open/close in 43.41 seconds: 103.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1980 (time 1781611813.26 total 10.00 last 197.95) - /unlink 4038 (time 1781611823.27 total 20.01 last 205.59) total: 4500 /unlink in 22.31 seconds: 201.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1169 (time 1781611838.19 total 10.01 last 116.81) - open/close 2279 (time 1781611848.19 total 20.01 last 110.94) - open/close 3510 (time 1781611858.20 total 30.02 last 123.06) total: 4500 open/close in 38.67 seconds: 116.36 ops/second - /unlink 2298 (time 1781611879.37 total 10.00 last 229.76) total: 4500 /unlink in 18.38 seconds: 244.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1676 (time 1781611900.27 total 10.00 last 167.57) - open/close 3340 (time 1781611910.28 total 20.01 last 166.29) total: 4500 open/close in 26.84 seconds: 167.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2802 (time 1781611929.09 total 10.00 last 280.14) total: 4500 /unlink in 16.49 seconds: 272.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1425 (time 1781611947.64 total 10.01 last 142.41) - open/close 2939 (time 1781611957.64 total 20.01 last 151.39) - open/close 4350 (time 1781611967.64 total 30.01 last 141.10) total: 4500 open/close in 31.15 seconds: 144.48 ops/second - /unlink 3079 (time 1781611980.84 total 10.00 last 307.83) total: 4500 /unlink in 15.43 seconds: 291.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1306 (time 1781611998.66 total 10.00 last 130.56) - open/close 2640 (time 1781612008.66 total 20.00 last 133.39) - open/close 4013 (time 1781612018.67 total 30.01 last 137.20) total: 4500 open/close in 33.55 seconds: 134.11 ops/second - /unlink 2851 (time 1781612034.08 total 10.00 last 285.10) total: 4500 /unlink in 15.91 seconds: 282.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1508 (time 1781612052.04 total 10.00 last 150.75) - open/close 2915 (time 1781612062.05 total 20.01 last 140.66) - open/close 4485 (time 1781612072.05 total 30.01 last 156.95) total: 4500 open/close in 30.08 seconds: 149.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 3316 (time 1781612083.85 total 10.00 last 331.50) total: 4500 /unlink in 13.67 seconds: 329.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1573 (time 1781612099.35 total 10.01 last 157.16) - open/close 3067 (time 1781612109.35 total 20.01 last 149.32) - open/close 4413 (time 1781612119.35 total 30.02 last 134.55) total: 4500 open/close in 30.69 seconds: 146.65 ops/second - /unlink 2864 (time 1781612132.20 total 10.00 last 286.32) total: 4500 /unlink in 16.43 seconds: 273.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1316 (time 1781612151.31 total 10.00 last 131.55) - open/close 2759 (time 1781612161.31 total 20.00 last 144.28) - open/close 4308 (time 1781612171.32 total 30.01 last 154.80) total: 4500 open/close in 31.32 seconds: 143.69 ops/second - /unlink 2889 (time 1781612184.55 total 10.00 last 288.90) total: 4500 /unlink in 15.62 seconds: 288.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1474 (time 1781612203.09 total 10.01 last 147.26) - open/close 2808 (time 1781612213.09 total 20.01 last 133.35) - open/close 4188 (time 1781612223.09 total 30.02 last 137.98) total: 4500 open/close in 32.20 seconds: 139.76 ops/second - /unlink 2683 (time 1781612237.28 total 10.00 last 268.20) total: 4500 /unlink in 16.15 seconds: 278.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1498 (time 1781612255.53 total 10.00 last 149.76) - open/close 3008 (time 1781612265.54 total 20.00 last 150.97) total: 4500 open/close in 29.64 seconds: 151.83 ops/second - /unlink 3735 (time 1781612286.99 total 10.00 last 373.49) total: 4500 /unlink in 12.25 seconds: 367.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1224 (time 1781612300.98 total 10.01 last 122.24) - open/close 2114 (time 1781612311.00 total 20.03 last 88.89) - open/close 3027 (time 1781612321.00 total 30.03 last 91.24) - open/close 4028 (time 1781612331.01 total 40.04 last 100.00) total: 4500 open/close in 44.61 seconds: 100.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2332 (time 1781612347.93 total 10.00 last 233.12) total: 4500 /unlink in 18.83 seconds: 238.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1302 (time 1781612369.10 total 10.01 last 130.12) - open/close 2488 (time 1781612379.10 total 20.01 last 118.58) - open/close 3693 (time 1781612389.11 total 30.01 last 120.46) total: 4500 open/close in 37.20 seconds: 120.96 ops/second - /unlink 2323 (time 1781612408.91 total 10.00 last 232.21) total: 4500 /unlink in 18.72 seconds: 240.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1234 (time 1781612429.97 total 10.01 last 123.27) - open/close 2455 (time 1781612439.97 total 20.01 last 122.09) - open/close 3626 (time 1781612449.97 total 30.02 last 117.04) total: 4500 open/close in 37.20 seconds: 120.96 ops/second - /unlink 2305 (time 1781612469.41 total 10.00 last 230.44) - /unlink 4251 (time 1781612479.42 total 20.01 last 194.55) total: 4500 /unlink in 20.98 seconds: 214.46 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11225: 94404 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg352-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg352-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg352-server stop mds service on oleg352-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg352-server stop mds service on oleg352-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg352-server unloading modules via unload_modules_local on: 'oleg352-server' oleg352-server: oleg352-server.virtnet: executing unload_modules_local oleg352-server: modules unloaded.