== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 15:18:08 (1787512688) Stopping clients: oleg344-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg344-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg344-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg344-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg344-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg344-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg344-server oleg344-server: oleg344-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg344-server' oleg344-server: oleg344-server.virtnet: executing load_modules_local oleg344-server: Loading modules from /home/green/git/lustre-release/lustre oleg344-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg344-server: detected 4 online CPUs by sysfs oleg344-server: MODOPTS_LIBCFS= oleg344-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg344-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg344-server: oleg344-server.virtnet: executing set_default_debug -1 all pdsh@oleg344-client: oleg344-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg344-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg344-server: oleg344-server.virtnet: executing set_default_debug -1 all pdsh@oleg344-client: oleg344-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg344-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg344-server: oleg344-server.virtnet: executing set_default_debug -1 all pdsh@oleg344-client: oleg344-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg344-client.virtnet: -o user_xattr,flock 192.168.203.144@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1312 (time 1787512835.49 total 10.00 last 131.15) - open/close 2621 (time 1787512845.49 total 20.01 last 130.81) - open/close 3821 (time 1787512855.49 total 30.01 last 119.99) total: 4500 open/close in 35.87 seconds: 125.46 ops/second - /unlink 1721 (time 1787512873.61 total 10.01 last 172.00) - /unlink 3740 (time 1787512883.62 total 20.01 last 201.85) total: 4500 /unlink in 23.43 seconds: 192.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1161 (time 1787512899.52 total 10.00 last 116.09) - open/close 2355 (time 1787512909.53 total 20.01 last 119.29) - open/close 3545 (time 1787512919.53 total 30.01 last 118.96) total: 4500 open/close in 38.31 seconds: 117.46 ops/second - /unlink 2308 (time 1787512940.10 total 10.00 last 230.79) - /unlink 4462 (time 1787512950.11 total 20.00 last 215.34) total: 4500 /unlink in 20.18 seconds: 223.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1175 (time 1787512963.00 total 10.00 last 117.47) - open/close 2512 (time 1787512973.01 total 20.01 last 133.66) - open/close 3742 (time 1787512983.01 total 30.01 last 122.93) total: 4500 open/close in 36.22 seconds: 124.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2503 (time 1787513001.36 total 10.00 last 250.26) total: 4500 /unlink in 18.38 seconds: 244.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1204 (time 1787513022.02 total 10.00 last 120.36) - open/close 2264 (time 1787513032.02 total 20.01 last 105.98) - open/close 3411 (time 1787513042.03 total 30.01 last 114.65) total: 4500 open/close in 39.09 seconds: 115.10 ops/second - /unlink 2449 (time 1787513063.45 total 10.00 last 244.85) total: 4500 /unlink in 16.72 seconds: 269.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1263 (time 1787513082.23 total 10.00 last 126.24) - open/close 2544 (time 1787513092.24 total 20.02 last 127.97) - open/close 3761 (time 1787513102.25 total 30.02 last 121.64) total: 4500 open/close in 36.06 seconds: 124.81 ops/second - /unlink 2487 (time 1787513120.55 total 10.00 last 248.67) total: 4500 /unlink in 18.37 seconds: 245.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1138 (time 1787513141.36 total 10.00 last 113.78) - open/close 2349 (time 1787513151.36 total 20.00 last 121.08) - open/close 3635 (time 1787513161.37 total 30.01 last 128.53) total: 4500 open/close in 37.34 seconds: 120.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2397 (time 1787513180.98 total 10.01 last 239.47) total: 4500 /unlink in 19.34 seconds: 232.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1308 (time 1787513202.86 total 10.00 last 130.76) - open/close 2809 (time 1787513212.87 total 20.01 last 150.06) - open/close 4233 (time 1787513222.87 total 30.01 last 142.32) total: 4500 open/close in 31.81 seconds: 141.46 ops/second - /unlink 2978 (time 1787513236.85 total 10.01 last 297.65) total: 4500 /unlink in 14.57 seconds: 308.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1567 (time 1787513253.16 total 10.00 last 156.68) - open/close 3006 (time 1787513263.17 total 20.01 last 143.81) - open/close 4400 (time 1787513273.18 total 30.01 last 139.31) total: 4500 open/close in 30.76 seconds: 146.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 3048 (time 1787513285.94 total 10.00 last 304.78) total: 4500 /unlink in 14.02 seconds: 320.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1952 (time 1787513301.94 total 10.00 last 195.14) - open/close 3515 (time 1787513311.94 total 20.01 last 156.26) total: 4500 open/close in 27.10 seconds: 166.05 ops/second - /unlink 3080 (time 1787513330.97 total 10.00 last 307.97) total: 4500 /unlink in 14.86 seconds: 302.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1611 (time 1787513347.70 total 10.01 last 161.00) - open/close 3314 (time 1787513357.70 total 20.01 last 170.28) total: 4500 open/close in 27.19 seconds: 165.53 ops/second - /unlink 3622 (time 1787513376.76 total 10.00 last 362.17) total: 4500 /unlink in 12.21 seconds: 368.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1811 (time 1787513390.92 total 10.00 last 181.04) - open/close 3950 (time 1787513400.92 total 20.00 last 213.89) total: 4500 open/close in 22.50 seconds: 200.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 total: 4500 /unlink in 9.14 seconds: 492.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 2180 (time 1787513425.62 total 10.00 last 217.92) total: 4500 open/close in 19.54 seconds: 230.35 ops/second total: 4500 /unlink in 9.11 seconds: 493.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1760 (time 1787513457.41 total 10.01 last 175.83) - open/close 3789 (time 1787513467.41 total 20.01 last 202.83) total: 4500 open/close in 23.48 seconds: 191.62 ops/second - /unlink 3813 (time 1787513482.55 total 10.00 last 381.20) total: 4500 /unlink in 11.45 seconds: 393.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 2245 (time 1787513495.77 total 10.00 last 224.44) total: 4500 open/close in 19.76 seconds: 227.79 ops/second - /unlink 4260 (time 1787513517.04 total 10.00 last 425.89) total: 4500 /unlink in 10.76 seconds: 418.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1540 (time 1787513529.68 total 10.01 last 153.84) - open/close 3021 (time 1787513539.68 total 20.01 last 148.06) total: 4500 open/close in 28.87 seconds: 155.88 ops/second - /unlink 3552 (time 1787513560.66 total 10.00 last 355.17) total: 4500 /unlink in 12.27 seconds: 366.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1910 (time 1787513574.66 total 10.01 last 190.88) - open/close 3742 (time 1787513584.67 total 20.01 last 183.16) total: 4500 open/close in 24.12 seconds: 186.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3440 (time 1787513600.61 total 10.00 last 343.93) total: 4500 /unlink in 13.52 seconds: 332.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1836 (time 1787513615.93 total 10.00 last 183.58) - open/close 4140 (time 1787513625.93 total 20.00 last 230.40) total: 4500 open/close in 21.71 seconds: 207.32 ops/second total: 4500 /unlink in 9.26 seconds: 486.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1856 (time 1787513650.08 total 10.00 last 185.60) - open/close 3611 (time 1787513660.08 total 20.00 last 175.42) total: 4500 open/close in 25.38 seconds: 177.30 ops/second - /unlink 3118 (time 1787513677.26 total 10.00 last 311.78) total: 4500 /unlink in 13.53 seconds: 332.52 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 94939 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg344-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg344-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg344-server stop mds service on oleg344-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg344-server stop mds service on oleg344-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg344-server unloading modules via unload_modules_local on: 'oleg344-server' oleg344-server: oleg344-server.virtnet: executing unload_modules_local oleg344-server: modules unloaded.