== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 16:45:03 (1763156703) Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg124-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg124-server oleg124-server: oleg124-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg124-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1309 (time 1763156837.74 total 10.01 last 130.82) - open/close 2798 (time 1763156847.74 total 20.01 last 148.88) - open/close 3973 (time 1763156857.75 total 30.02 last 117.40) total: 4500 open/close in 35.20 seconds: 127.83 ops/second - /unlink 2082 (time 1763156875.32 total 10.01 last 208.10) - /unlink 4361 (time 1763156885.32 total 20.01 last 227.84) total: 4500 /unlink in 20.58 seconds: 218.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1435 (time 1763156898.07 total 10.00 last 143.45) - open/close 2922 (time 1763156908.07 total 20.00 last 148.70) total: 4500 open/close in 29.30 seconds: 153.58 ops/second - /unlink 2447 (time 1763156929.11 total 10.00 last 244.68) total: 4500 /unlink in 18.32 seconds: 245.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1255 (time 1763156949.37 total 10.00 last 125.46) - open/close 3024 (time 1763156959.37 total 20.01 last 176.84) total: 4500 open/close in 28.65 seconds: 157.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 3337 (time 1763156979.82 total 10.00 last 333.65) total: 4500 /unlink in 13.80 seconds: 326.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1640 (time 1763156995.46 total 10.01 last 163.88) - open/close 3086 (time 1763157005.46 total 20.01 last 144.59) total: 4500 open/close in 28.22 seconds: 159.49 ops/second - /unlink 3573 (time 1763157025.33 total 10.00 last 357.26) total: 4500 /unlink in 12.63 seconds: 356.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1867 (time 1763157039.79 total 10.00 last 186.68) - open/close 3645 (time 1763157049.79 total 20.00 last 177.80) total: 4500 open/close in 23.94 seconds: 187.93 ops/second - /unlink 3199 (time 1763157065.47 total 10.00 last 319.87) total: 4500 /unlink in 14.20 seconds: 316.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1350 (time 1763157081.71 total 10.00 last 134.94) - open/close 2897 (time 1763157091.72 total 20.01 last 154.62) - open/close 4349 (time 1763157101.72 total 30.01 last 145.17) total: 4500 open/close in 30.88 seconds: 145.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 3005 (time 1763157114.38 total 10.00 last 300.49) total: 4500 /unlink in 15.12 seconds: 297.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1423 (time 1763157131.65 total 10.00 last 142.29) - open/close 2922 (time 1763157141.65 total 20.00 last 149.90) total: 4500 open/close in 29.20 seconds: 154.11 ops/second - /unlink 2856 (time 1763157162.73 total 10.00 last 285.56) total: 4500 /unlink in 14.61 seconds: 307.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1880 (time 1763157179.11 total 10.00 last 187.94) - open/close 3322 (time 1763157189.11 total 20.00 last 144.19) total: 4500 open/close in 29.31 seconds: 153.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2903 (time 1763157210.59 total 10.00 last 290.28) total: 4500 /unlink in 14.66 seconds: 306.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 2054 (time 1763157226.99 total 10.00 last 205.33) - open/close 3829 (time 1763157236.99 total 20.01 last 177.43) total: 4500 open/close in 23.20 seconds: 193.96 ops/second - /unlink 3033 (time 1763157251.93 total 10.00 last 303.27) total: 4500 /unlink in 14.58 seconds: 308.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1736 (time 1763157268.36 total 10.01 last 173.50) - open/close 3392 (time 1763157278.36 total 20.01 last 165.60) total: 4500 open/close in 27.28 seconds: 164.94 ops/second - /unlink 3050 (time 1763157297.43 total 10.00 last 304.99) total: 4500 /unlink in 14.66 seconds: 306.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 2059 (time 1763157314.01 total 10.00 last 205.83) - open/close 3885 (time 1763157324.01 total 20.00 last 182.58) total: 4500 open/close in 24.71 seconds: 182.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 3633 (time 1763157340.64 total 10.00 last 363.19) total: 4500 /unlink in 12.72 seconds: 353.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1823 (time 1763157355.14 total 10.00 last 182.27) - open/close 3665 (time 1763157365.15 total 20.01 last 184.09) total: 4500 open/close in 24.25 seconds: 185.57 ops/second - /unlink 3694 (time 1763157381.10 total 10.00 last 369.37) total: 4500 /unlink in 11.65 seconds: 386.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 2418 (time 1763157394.42 total 10.00 last 241.73) - open/close 4161 (time 1763157404.42 total 20.01 last 174.26) total: 4500 open/close in 21.37 seconds: 210.57 ops/second total: 4500 /unlink in 9.53 seconds: 472.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1660 (time 1763157428.56 total 10.00 last 165.97) - open/close 3275 (time 1763157438.57 total 20.01 last 161.38) total: 4500 open/close in 27.93 seconds: 161.10 ops/second - /unlink 3695 (time 1763157458.24 total 10.00 last 369.46) total: 4500 /unlink in 11.73 seconds: 383.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1679 (time 1763157471.63 total 10.00 last 167.85) - open/close 3265 (time 1763157481.63 total 20.00 last 158.58) total: 4500 open/close in 27.11 seconds: 165.99 ops/second - /unlink 4138 (time 1763157500.53 total 10.00 last 413.79) total: 4500 /unlink in 11.03 seconds: 408.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 2239 (time 1763157513.21 total 10.00 last 223.88) total: 4500 open/close in 19.96 seconds: 225.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 4345 (time 1763157534.99 total 10.00 last 434.42) total: 4500 /unlink in 10.49 seconds: 428.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1873 (time 1763157547.24 total 10.00 last 187.28) - open/close 3516 (time 1763157557.24 total 20.00 last 164.29) total: 4500 open/close in 24.95 seconds: 180.34 ops/second - /unlink 3372 (time 1763157573.89 total 10.00 last 337.08) total: 4500 /unlink in 13.15 seconds: 342.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 2204 (time 1763157588.81 total 10.00 last 220.34) total: 4500 open/close in 19.52 seconds: 230.53 ops/second total: 4500 /unlink in 9.54 seconds: 471.50 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11098: 93890 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing unload_modules_local oleg124-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded.