== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:29:19 (1763353759) Stopping clients: oleg130-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg130-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg130-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg130-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg130-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg130-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg130-server oleg130-server: oleg130-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg130-server' oleg130-server: oleg130-server.virtnet: executing load_modules_local oleg130-server: Loading modules from /home/green/git/lustre-release/lustre oleg130-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg130-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg130-server: detected 4 online CPUs by sysfs oleg130-server: MODOPTS_LIBCFS= oleg130-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg130-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg130-server: oleg130-server.virtnet: executing set_default_debug -1 all pdsh@oleg130-client: oleg130-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg130-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg130-server: oleg130-server.virtnet: executing set_default_debug -1 all pdsh@oleg130-client: oleg130-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg130-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg130-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg130-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg130-server: oleg130-server.virtnet: executing set_default_debug -1 all pdsh@oleg130-client: oleg130-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg130-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg130-client.virtnet: -o user_xattr,flock 192.168.201.130@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1595 (time 1763353874.31 total 10.00 last 159.48) - open/close 3116 (time 1763353884.31 total 20.00 last 152.05) total: 4500 open/close in 29.48 seconds: 152.65 ops/second - /unlink 3163 (time 1763353905.84 total 10.00 last 316.24) total: 4500 /unlink in 14.29 seconds: 315.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1363 (time 1763353921.95 total 10.01 last 136.21) - open/close 2849 (time 1763353931.96 total 20.01 last 148.55) - open/close 4374 (time 1763353941.96 total 30.01 last 152.50) total: 4500 open/close in 31.02 seconds: 145.07 ops/second - /unlink 4317 (time 1763353954.85 total 10.00 last 431.57) total: 4500 /unlink in 10.55 seconds: 426.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 2568 (time 1763353967.18 total 10.00 last 256.72) total: 4500 open/close in 19.56 seconds: 230.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 4246 (time 1763353988.36 total 10.00 last 424.58) total: 4500 /unlink in 10.62 seconds: 423.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 2068 (time 1763354000.57 total 10.00 last 206.73) - open/close 3571 (time 1763354010.57 total 20.01 last 150.24) total: 4500 open/close in 25.68 seconds: 175.23 ops/second - /unlink 3045 (time 1763354028.01 total 10.00 last 304.48) total: 4500 /unlink in 14.18 seconds: 317.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1795 (time 1763354043.99 total 10.00 last 179.45) - open/close 3491 (time 1763354054.00 total 20.01 last 169.52) total: 4500 open/close in 25.49 seconds: 176.51 ops/second - /unlink 3687 (time 1763354071.14 total 10.00 last 368.60) total: 4500 /unlink in 12.29 seconds: 366.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1531 (time 1763354085.18 total 10.00 last 153.07) - open/close 3271 (time 1763354095.18 total 20.01 last 173.94) total: 4500 open/close in 27.38 seconds: 164.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2868 (time 1763354114.44 total 10.00 last 286.77) total: 4500 /unlink in 15.47 seconds: 290.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1514 (time 1763354131.72 total 10.00 last 151.33) - open/close 3367 (time 1763354141.72 total 20.01 last 185.29) total: 4500 open/close in 26.55 seconds: 169.48 ops/second - /unlink 2864 (time 1763354160.19 total 10.00 last 286.38) total: 4500 /unlink in 14.85 seconds: 303.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1708 (time 1763354176.81 total 10.01 last 170.71) - open/close 3422 (time 1763354186.81 total 20.01 last 171.34) total: 4500 open/close in 26.53 seconds: 169.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 3369 (time 1763354205.23 total 10.00 last 336.87) total: 4500 /unlink in 13.04 seconds: 345.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1646 (time 1763354220.03 total 10.00 last 164.53) - open/close 3534 (time 1763354230.03 total 20.01 last 188.77) total: 4500 open/close in 25.86 seconds: 174.00 ops/second - /unlink 3108 (time 1763354247.70 total 10.00 last 310.76) total: 4500 /unlink in 15.09 seconds: 298.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1840 (time 1763354264.67 total 10.00 last 183.91) - open/close 3514 (time 1763354274.67 total 20.01 last 167.32) total: 4500 open/close in 25.92 seconds: 173.62 ops/second - /unlink 3047 (time 1763354292.34 total 10.00 last 304.60) total: 4500 /unlink in 14.64 seconds: 307.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1553 (time 1763354308.78 total 10.00 last 155.23) - open/close 3042 (time 1763354318.79 total 20.01 last 148.82) - open/close 4300 (time 1763354328.80 total 30.02 last 125.71) total: 4500 open/close in 31.42 seconds: 143.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2533 (time 1763354342.09 total 10.00 last 253.28) total: 4500 /unlink in 18.67 seconds: 241.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1346 (time 1763354362.67 total 10.02 last 134.37) - open/close 2910 (time 1763354372.68 total 20.03 last 156.18) total: 4500 open/close in 28.90 seconds: 155.71 ops/second - /unlink 3205 (time 1763354393.46 total 10.00 last 320.47) total: 4500 /unlink in 13.20 seconds: 340.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 2185 (time 1763354408.39 total 10.00 last 218.44) total: 4500 open/close in 17.99 seconds: 250.14 ops/second - /unlink 3352 (time 1763354428.00 total 10.00 last 335.15) total: 4500 /unlink in 13.22 seconds: 340.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 2037 (time 1763354442.97 total 10.00 last 203.63) - open/close 3938 (time 1763354452.98 total 20.01 last 190.02) total: 4500 open/close in 23.63 seconds: 190.47 ops/second - /unlink 3293 (time 1763354468.50 total 10.00 last 329.26) total: 4500 /unlink in 14.25 seconds: 315.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 2169 (time 1763354484.61 total 10.00 last 216.81) - open/close 4175 (time 1763354494.61 total 20.01 last 200.57) total: 4500 open/close in 21.43 seconds: 210.00 ops/second - /unlink 3911 (time 1763354507.63 total 10.00 last 391.09) total: 4500 /unlink in 11.38 seconds: 395.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 2003 (time 1763354520.65 total 10.00 last 200.26) - open/close 4057 (time 1763354530.65 total 20.00 last 205.35) total: 4500 open/close in 22.00 seconds: 204.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3500 (time 1763354544.41 total 10.00 last 349.99) total: 4500 /unlink in 13.02 seconds: 345.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1687 (time 1763354559.27 total 10.00 last 168.63) - open/close 3278 (time 1763354569.27 total 20.01 last 159.07) total: 4500 open/close in 28.53 seconds: 157.72 ops/second - /unlink 2976 (time 1763354589.68 total 10.00 last 297.57) total: 4500 /unlink in 14.45 seconds: 311.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1150 (time 1763354607.22 total 10.00 last 114.96) - open/close 2565 (time 1763354617.22 total 20.00 last 141.50) - open/close 4373 (time 1763354627.22 total 30.01 last 180.72) total: 4500 open/close in 30.63 seconds: 146.91 ops/second - /unlink 3085 (time 1763354639.65 total 10.00 last 308.48) total: 4500 /unlink in 14.33 seconds: 314.01 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11098: 93443 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg130-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg130-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg130-server stop mds service on oleg130-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg130-server stop mds service on oleg130-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg130-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg130-server' oleg130-server: oleg130-server.virtnet: executing unload_modules_local oleg130-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg130-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded.