== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:34:05 (1783564445) Stopping clients: oleg415-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg415-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg415-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg415-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg415-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg415-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg415-server oleg415-server: oleg415-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: MODOPTS_LIBCFS= oleg415-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg415-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg415-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg415-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg415-client.virtnet: -o user_xattr,flock 192.168.204.115@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1015 (time 1783564595.35 total 10.01 last 101.40) - open/close 1877 (time 1783564605.36 total 20.02 last 86.14) - open/close 2772 (time 1783564615.37 total 30.03 last 89.41) - open/close 3715 (time 1783564625.37 total 40.03 last 94.29) total: 4500 open/close in 49.25 seconds: 91.37 ops/second - /unlink 1625 (time 1783564647.54 total 10.00 last 162.45) - /unlink 3344 (time 1783564657.54 total 20.01 last 171.83) total: 4500 /unlink in 26.47 seconds: 169.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 953 (time 1783564676.51 total 10.00 last 95.27) - open/close 1908 (time 1783564686.51 total 20.01 last 95.45) - open/close 2925 (time 1783564696.51 total 30.01 last 101.67) - open/close 4004 (time 1783564706.51 total 40.01 last 107.89) total: 4500 open/close in 44.61 seconds: 100.88 ops/second - /unlink 1819 (time 1783564723.57 total 10.00 last 181.83) - /unlink 3752 (time 1783564733.57 total 20.01 last 193.26) total: 4500 /unlink in 24.22 seconds: 185.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1011 (time 1783564751.11 total 10.01 last 101.01) - open/close 2208 (time 1783564761.11 total 20.01 last 119.70) - open/close 3392 (time 1783564771.11 total 30.02 last 118.33) - open/close 4416 (time 1783564781.12 total 40.03 last 102.30) total: 4500 open/close in 40.92 seconds: 109.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2017 (time 1783564794.68 total 10.00 last 201.62) - /unlink 3947 (time 1783564804.68 total 20.00 last 192.98) total: 4500 /unlink in 22.77 seconds: 197.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1065 (time 1783564819.63 total 10.00 last 106.50) - open/close 2325 (time 1783564829.63 total 20.00 last 125.97) - open/close 3613 (time 1783564839.64 total 30.01 last 128.77) total: 4500 open/close in 37.78 seconds: 119.12 ops/second - /unlink 2414 (time 1783564859.55 total 10.00 last 241.34) total: 4500 /unlink in 17.86 seconds: 251.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1230 (time 1783564879.65 total 10.00 last 122.95) - open/close 2465 (time 1783564889.65 total 20.01 last 123.48) - open/close 3714 (time 1783564899.66 total 30.01 last 124.82) total: 4500 open/close in 36.15 seconds: 124.47 ops/second - /unlink 2294 (time 1783564917.80 total 10.00 last 229.40) total: 4500 /unlink in 19.93 seconds: 225.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1128 (time 1783564940.38 total 10.00 last 112.74) - open/close 2183 (time 1783564950.38 total 20.01 last 105.44) - open/close 3375 (time 1783564960.39 total 30.02 last 119.13) - open/close 4479 (time 1783564970.39 total 40.02 last 110.35) total: 4500 open/close in 40.20 seconds: 111.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2239 (time 1783564982.78 total 10.00 last 223.87) - /unlink 4095 (time 1783564992.78 total 20.00 last 185.54) total: 4500 /unlink in 22.23 seconds: 202.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1270 (time 1783565007.30 total 10.01 last 126.85) - open/close 2506 (time 1783565017.30 total 20.01 last 123.59) - open/close 3614 (time 1783565027.31 total 30.02 last 110.77) total: 4500 open/close in 39.10 seconds: 115.09 ops/second - /unlink 2134 (time 1783565049.00 total 10.00 last 213.37) - /unlink 4358 (time 1783565059.00 total 20.00 last 222.38) total: 4500 /unlink in 20.66 seconds: 217.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1164 (time 1783565071.92 total 10.00 last 116.34) - open/close 2269 (time 1783565081.92 total 20.01 last 110.46) - open/close 3243 (time 1783565091.93 total 30.01 last 97.35) - open/close 4247 (time 1783565101.93 total 40.02 last 100.36) total: 4500 open/close in 42.25 seconds: 106.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2277 (time 1783565116.38 total 10.00 last 227.64) - /unlink 4491 (time 1783565126.39 total 20.01 last 221.20) total: 4500 /unlink in 20.05 seconds: 224.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1091 (time 1783565138.58 total 10.01 last 109.03) - open/close 2372 (time 1783565148.59 total 20.01 last 128.02) - open/close 3726 (time 1783565158.59 total 30.02 last 135.32) total: 4500 open/close in 36.71 seconds: 122.60 ops/second - /unlink 2308 (time 1783565177.42 total 10.00 last 230.76) total: 4500 /unlink in 19.32 seconds: 232.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1094 (time 1783565198.67 total 10.00 last 109.39) - open/close 2073 (time 1783565208.67 total 20.00 last 97.90) - open/close 3140 (time 1783565218.67 total 30.00 last 106.67) - open/close 4373 (time 1783565228.67 total 40.01 last 123.26) total: 4500 open/close in 41.03 seconds: 109.67 ops/second - /unlink 2221 (time 1783565241.86 total 10.00 last 222.06) total: 4500 /unlink in 19.99 seconds: 225.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1416 (time 1783565263.71 total 10.00 last 141.56) - open/close 3271 (time 1783565273.71 total 20.00 last 185.47) total: 4500 open/close in 27.08 seconds: 166.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 3325 (time 1783565292.45 total 10.00 last 332.50) total: 4500 /unlink in 13.66 seconds: 329.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1664 (time 1783565308.06 total 10.00 last 166.37) - open/close 3393 (time 1783565318.07 total 20.01 last 172.84) total: 4500 open/close in 26.98 seconds: 166.76 ops/second - /unlink 4341 (time 1783565336.80 total 10.00 last 433.90) total: 4500 /unlink in 10.45 seconds: 430.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 2038 (time 1783565348.97 total 10.00 last 203.80) - open/close 4289 (time 1783565358.98 total 20.00 last 225.03) total: 4500 open/close in 21.00 seconds: 214.30 ops/second - /unlink 4000 (time 1783565371.84 total 10.00 last 399.90) total: 4500 /unlink in 11.53 seconds: 390.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1982 (time 1783565385.20 total 10.00 last 198.12) - open/close 3928 (time 1783565395.21 total 20.01 last 194.46) total: 4500 open/close in 23.93 seconds: 188.09 ops/second - /unlink 4250 (time 1783565410.99 total 10.00 last 424.99) total: 4500 /unlink in 10.61 seconds: 423.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 2248 (time 1783565423.29 total 10.00 last 224.71) - open/close 4461 (time 1783565433.29 total 20.01 last 221.20) total: 4500 open/close in 20.22 seconds: 222.54 ops/second - /unlink 3628 (time 1783565445.23 total 10.00 last 362.78) total: 4500 /unlink in 12.03 seconds: 374.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1788 (time 1783565459.01 total 10.00 last 178.73) - open/close 3902 (time 1783565469.02 total 20.01 last 211.35) total: 4500 open/close in 23.29 seconds: 193.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3823 (time 1783565484.22 total 10.00 last 382.25) total: 4500 /unlink in 11.33 seconds: 397.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 2575 (time 1783565497.23 total 10.01 last 257.32) total: 4500 open/close in 19.34 seconds: 232.71 ops/second total: 4500 /unlink in 9.05 seconds: 497.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 2310 (time 1783565528.73 total 10.01 last 230.88) - open/close 4324 (time 1783565538.73 total 20.01 last 201.34) total: 4500 open/close in 21.05 seconds: 213.80 ops/second - /unlink 4478 (time 1783565551.48 total 10.00 last 447.68) total: 4500 /unlink in 10.05 seconds: 447.85 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11242: 94182 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg415-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg415-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg415-server unloading modules via unload_modules_local on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing unload_modules_local oleg415-server: modules unloaded.