== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 12:45:44 (1772473544) Stopping clients: oleg316-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg316-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg316-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg316-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg316-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg316-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg316-server oleg316-server: oleg316-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg316-server' oleg316-server: oleg316-server.virtnet: executing load_modules_local oleg316-server: Loading modules from /home/green/git/lustre-release/lustre oleg316-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg316-server: detected 4 online CPUs by sysfs oleg316-server: MODOPTS_LIBCFS= oleg316-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg316-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug -1 all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg316-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg316-server: oleg316-server.virtnet: executing set_default_debug -1 all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg316-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg316-server: oleg316-server.virtnet: executing set_default_debug -1 all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1165 (time 1772473699.95 total 10.00 last 116.50) - open/close 2265 (time 1772473709.95 total 20.00 last 109.95) - open/close 3378 (time 1772473719.95 total 30.01 last 111.29) - open/close 4390 (time 1772473729.95 total 40.01 last 101.19) total: 4500 open/close in 40.96 seconds: 109.86 ops/second - /unlink 1937 (time 1772473743.98 total 10.00 last 193.62) - /unlink 3850 (time 1772473753.98 total 20.00 last 191.30) total: 4500 /unlink in 24.12 seconds: 186.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1080 (time 1772473771.49 total 10.00 last 107.97) - open/close 2099 (time 1772473781.50 total 20.02 last 101.77) - open/close 3156 (time 1772473791.51 total 30.02 last 105.60) - open/close 4214 (time 1772473801.52 total 40.03 last 105.74) total: 4500 open/close in 42.57 seconds: 105.70 ops/second - /unlink 1606 (time 1772473817.37 total 10.00 last 160.57) - /unlink 3583 (time 1772473827.37 total 20.00 last 197.66) total: 4500 /unlink in 24.78 seconds: 181.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 962 (time 1772473845.99 total 10.01 last 96.11) - open/close 1893 (time 1772473855.99 total 20.01 last 93.09) - open/close 2844 (time 1772473865.99 total 30.01 last 95.09) - open/close 3954 (time 1772473875.99 total 40.02 last 110.97) total: 4500 open/close in 44.03 seconds: 102.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2009 (time 1772473892.50 total 10.00 last 200.88) - /unlink 3969 (time 1772473902.50 total 20.00 last 195.98) total: 4500 /unlink in 22.57 seconds: 199.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1156 (time 1772473917.83 total 10.00 last 115.56) - open/close 2064 (time 1772473927.84 total 20.02 last 90.69) - open/close 3007 (time 1772473937.84 total 30.02 last 94.28) - open/close 4025 (time 1772473947.86 total 40.04 last 101.61) total: 4500 open/close in 44.83 seconds: 100.37 ops/second - /unlink 1731 (time 1772473967.09 total 10.00 last 173.07) - /unlink 3714 (time 1772473977.09 total 20.00 last 198.29) total: 4500 /unlink in 24.16 seconds: 186.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1104 (time 1772473994.92 total 10.00 last 110.35) - open/close 2184 (time 1772474004.92 total 20.00 last 108.00) - open/close 3268 (time 1772474014.93 total 30.01 last 108.34) - open/close 4396 (time 1772474024.93 total 40.01 last 112.78) total: 4500 open/close in 40.97 seconds: 109.84 ops/second - /unlink 1925 (time 1772474038.92 total 10.00 last 192.42) - /unlink 3901 (time 1772474048.93 total 20.01 last 197.48) total: 4500 /unlink in 23.00 seconds: 195.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1027 (time 1772474065.31 total 10.00 last 102.69) - open/close 2035 (time 1772474075.32 total 20.01 last 100.76) - open/close 3074 (time 1772474085.32 total 30.01 last 103.89) - open/close 4250 (time 1772474095.32 total 40.01 last 117.56) total: 4500 open/close in 42.15 seconds: 106.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1962 (time 1772474110.31 total 10.00 last 196.13) - /unlink 3967 (time 1772474120.31 total 20.01 last 200.47) total: 4500 /unlink in 22.41 seconds: 200.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1181 (time 1772474135.48 total 10.01 last 118.00) - open/close 2361 (time 1772474145.48 total 20.01 last 117.99) - open/close 3365 (time 1772474155.48 total 30.01 last 100.37) - open/close 4390 (time 1772474165.50 total 40.03 last 102.30) total: 4500 open/close in 41.00 seconds: 109.77 ops/second - /unlink 2085 (time 1772474179.30 total 10.00 last 208.46) - /unlink 4290 (time 1772474189.31 total 20.01 last 220.42) total: 4500 /unlink in 20.71 seconds: 217.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1107 (time 1772474203.06 total 10.00 last 110.68) - open/close 2106 (time 1772474213.07 total 20.01 last 99.84) - open/close 3234 (time 1772474223.07 total 30.01 last 112.80) - open/close 4429 (time 1772474233.08 total 40.02 last 119.41) total: 4500 open/close in 40.54 seconds: 110.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1979 (time 1772474246.25 total 10.00 last 197.86) - /unlink 4300 (time 1772474256.25 total 20.00 last 232.05) total: 4500 /unlink in 21.03 seconds: 213.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1129 (time 1772474269.73 total 10.01 last 112.83) - open/close 2289 (time 1772474279.73 total 20.01 last 115.99) - open/close 3448 (time 1772474289.74 total 30.01 last 115.84) total: 4500 open/close in 39.36 seconds: 114.32 ops/second - /unlink 2460 (time 1772474311.28 total 10.00 last 245.90) total: 4500 /unlink in 19.26 seconds: 233.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1011 (time 1772474334.15 total 10.02 last 100.94) - open/close 2228 (time 1772474344.15 total 20.02 last 121.69) - open/close 3419 (time 1772474354.15 total 30.02 last 119.05) total: 4500 open/close in 39.36 seconds: 114.34 ops/second - /unlink 2233 (time 1772474376.32 total 10.00 last 223.26) - /unlink 4229 (time 1772474386.32 total 20.00 last 199.58) total: 4500 /unlink in 21.29 seconds: 211.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1210 (time 1772474400.02 total 10.00 last 120.99) - open/close 2439 (time 1772474410.03 total 20.01 last 122.85) - open/close 3654 (time 1772474420.03 total 30.01 last 121.50) total: 4500 open/close in 36.55 seconds: 123.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2377 (time 1772474438.68 total 10.00 last 237.68) total: 4500 /unlink in 19.19 seconds: 234.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1210 (time 1772474460.70 total 10.01 last 120.92) - open/close 2463 (time 1772474470.71 total 20.01 last 125.22) - open/close 3771 (time 1772474480.71 total 30.02 last 130.72) total: 4500 open/close in 35.21 seconds: 127.81 ops/second - /unlink 2609 (time 1772474498.25 total 10.00 last 260.87) total: 4500 /unlink in 17.42 seconds: 258.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1247 (time 1772474518.30 total 10.01 last 124.54) - open/close 2457 (time 1772474528.30 total 20.01 last 120.99) - open/close 3806 (time 1772474538.31 total 30.02 last 134.79) total: 4500 open/close in 35.39 seconds: 127.15 ops/second - /unlink 2492 (time 1772474556.06 total 10.00 last 249.19) total: 4500 /unlink in 17.53 seconds: 256.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1437 (time 1772474575.64 total 10.00 last 143.64) - open/close 2874 (time 1772474585.64 total 20.01 last 143.63) - open/close 4422 (time 1772474595.65 total 30.01 last 154.75) total: 4500 open/close in 30.55 seconds: 147.28 ops/second - /unlink 2676 (time 1772474608.20 total 10.00 last 267.57) total: 4500 /unlink in 17.64 seconds: 255.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1561 (time 1772474628.10 total 10.00 last 156.09) - open/close 2972 (time 1772474638.11 total 20.00 last 141.05) - open/close 4464 (time 1772474648.11 total 30.01 last 149.12) total: 4500 open/close in 30.28 seconds: 148.60 ops/second - /unlink 3294 (time 1772474660.07 total 10.00 last 329.30) total: 4500 /unlink in 14.71 seconds: 305.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1358 (time 1772474676.75 total 10.00 last 135.76) - open/close 2688 (time 1772474686.76 total 20.01 last 132.87) - open/close 4068 (time 1772474696.77 total 30.02 last 137.90) total: 4500 open/close in 33.04 seconds: 136.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2735 (time 1772474711.96 total 10.00 last 273.49) total: 4500 /unlink in 16.75 seconds: 268.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1617 (time 1772474730.81 total 10.02 last 161.42) - open/close 3244 (time 1772474740.81 total 20.02 last 162.64) total: 4500 open/close in 27.74 seconds: 162.23 ops/second - /unlink 2538 (time 1772474760.49 total 10.00 last 253.72) total: 4500 /unlink in 17.30 seconds: 260.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1314 (time 1772474780.30 total 10.00 last 131.37) - open/close 2633 (time 1772474790.31 total 20.01 last 131.83) - open/close 3957 (time 1772474800.31 total 30.01 last 132.33) total: 4500 open/close in 33.86 seconds: 132.90 ops/second - /unlink 2595 (time 1772474816.41 total 10.00 last 259.46) total: 4500 /unlink in 17.74 seconds: 253.69 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11170: 94178 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg316-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg316-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg316-server stop mds service on oleg316-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg316-server stop mds service on oleg316-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg316-server unloading modules via unload_modules_local on: 'oleg316-server' oleg316-server: oleg316-server.virtnet: executing unload_modules_local oleg316-server: modules unloaded.