== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 15:30:19 (1788809419) Stopping clients: oleg139-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg139-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg139-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg139-server oleg139-server: oleg139-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing load_modules_local oleg139-server: Loading modules from /home/green/git/lustre-release/lustre oleg139-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg139-server: detected 4 online CPUs by sysfs oleg139-server: MODOPTS_LIBCFS= oleg139-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg139-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg139-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1003 (time 1788809586.56 total 10.01 last 100.25) - open/close 1994 (time 1788809596.56 total 20.01 last 99.04) - open/close 2909 (time 1788809606.57 total 30.02 last 91.43) - open/close 3884 (time 1788809616.58 total 40.03 last 97.43) total: 4500 open/close in 45.14 seconds: 99.69 ops/second - /unlink 1886 (time 1788809634.82 total 10.00 last 188.52) - /unlink 3762 (time 1788809644.83 total 20.01 last 187.54) total: 4500 /unlink in 23.85 seconds: 188.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1122 (time 1788809661.67 total 10.01 last 112.13) - open/close 2053 (time 1788809671.67 total 20.01 last 93.10) - open/close 3027 (time 1788809681.68 total 30.01 last 97.33) - open/close 3910 (time 1788809691.69 total 40.02 last 88.24) total: 4500 open/close in 46.60 seconds: 96.57 ops/second - /unlink 1854 (time 1788809711.27 total 10.01 last 185.31) - /unlink 3746 (time 1788809721.27 total 20.01 last 189.13) total: 4500 /unlink in 23.36 seconds: 192.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 938 (time 1788809738.40 total 10.02 last 93.65) - open/close 1801 (time 1788809748.40 total 20.02 last 86.24) - open/close 2637 (time 1788809758.41 total 30.03 last 83.50) - open/close 3544 (time 1788809768.44 total 40.06 last 90.50) - open/close 4457 (time 1788809778.44 total 50.06 last 91.27) total: 4500 open/close in 50.47 seconds: 89.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2104 (time 1788809791.91 total 10.00 last 210.37) - /unlink 4059 (time 1788809801.92 total 20.00 last 195.47) total: 4500 /unlink in 21.82 seconds: 206.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1038 (time 1788809816.63 total 10.01 last 103.74) - open/close 2069 (time 1788809826.64 total 20.02 last 102.99) - open/close 3137 (time 1788809836.65 total 30.02 last 106.72) - open/close 4099 (time 1788809846.65 total 40.03 last 96.16) total: 4500 open/close in 43.69 seconds: 102.99 ops/second - /unlink 1820 (time 1788809863.15 total 10.01 last 181.88) - /unlink 3725 (time 1788809873.16 total 20.01 last 190.37) total: 4500 /unlink in 23.98 seconds: 187.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1083 (time 1788809890.39 total 10.00 last 108.30) - open/close 2092 (time 1788809900.40 total 20.01 last 100.80) - open/close 3276 (time 1788809910.40 total 30.01 last 118.36) - open/close 4365 (time 1788809920.40 total 40.01 last 108.89) total: 4500 open/close in 41.54 seconds: 108.34 ops/second - /unlink 2206 (time 1788809934.64 total 10.01 last 220.40) - /unlink 4229 (time 1788809944.64 total 20.01 last 202.28) total: 4500 /unlink in 21.30 seconds: 211.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1072 (time 1788809958.66 total 10.00 last 107.16) - open/close 2220 (time 1788809968.66 total 20.01 last 114.78) - open/close 3385 (time 1788809978.67 total 30.02 last 116.37) - open/close 4441 (time 1788809988.67 total 40.02 last 105.55) total: 4500 open/close in 40.64 seconds: 110.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1983 (time 1788810001.94 total 10.00 last 198.24) - /unlink 3962 (time 1788810011.94 total 20.00 last 197.86) total: 4500 /unlink in 23.04 seconds: 195.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 983 (time 1788810028.17 total 10.00 last 98.26) - open/close 1949 (time 1788810038.18 total 20.01 last 96.51) - open/close 2919 (time 1788810048.19 total 30.02 last 96.93) - open/close 3958 (time 1788810058.20 total 40.03 last 103.81) total: 4500 open/close in 45.79 seconds: 98.28 ops/second - /unlink 2175 (time 1788810076.87 total 10.00 last 217.49) - /unlink 4110 (time 1788810086.87 total 20.00 last 193.48) total: 4500 /unlink in 22.26 seconds: 202.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1008 (time 1788810102.04 total 10.01 last 100.74) - open/close 2064 (time 1788810112.04 total 20.01 last 105.58) - open/close 3021 (time 1788810122.04 total 30.01 last 95.66) - open/close 4089 (time 1788810132.05 total 40.01 last 106.78) total: 4500 open/close in 43.73 seconds: 102.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1945 (time 1788810148.43 total 10.00 last 194.47) - /unlink 3865 (time 1788810158.44 total 20.00 last 191.98) total: 4500 /unlink in 23.41 seconds: 192.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1008 (time 1788810174.88 total 10.00 last 100.76) - open/close 2108 (time 1788810184.89 total 20.01 last 109.93) - open/close 3233 (time 1788810194.90 total 30.02 last 112.43) - open/close 4297 (time 1788810204.90 total 40.03 last 106.31) total: 4500 open/close in 41.69 seconds: 107.94 ops/second - /unlink 2349 (time 1788810218.72 total 10.00 last 234.80) total: 4500 /unlink in 18.99 seconds: 236.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1300 (time 1788810239.84 total 10.01 last 129.92) - open/close 2531 (time 1788810249.84 total 20.01 last 123.08) - open/close 3687 (time 1788810259.85 total 30.02 last 115.46) total: 4500 open/close in 37.13 seconds: 121.19 ops/second - /unlink 2197 (time 1788810279.31 total 10.01 last 219.56) total: 4500 /unlink in 19.19 seconds: 234.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1271 (time 1788810300.65 total 10.00 last 127.07) - open/close 2469 (time 1788810310.66 total 20.01 last 119.73) - open/close 3767 (time 1788810320.66 total 30.01 last 129.80) total: 4500 open/close in 35.77 seconds: 125.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2689 (time 1788810338.70 total 10.00 last 268.90) total: 4500 /unlink in 17.40 seconds: 258.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1307 (time 1788810358.53 total 10.01 last 130.62) - open/close 2640 (time 1788810368.54 total 20.01 last 133.24) - open/close 3854 (time 1788810378.54 total 30.01 last 121.40) total: 4500 open/close in 34.55 seconds: 130.24 ops/second - /unlink 2792 (time 1788810394.92 total 10.00 last 279.16) total: 4500 /unlink in 17.12 seconds: 262.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1480 (time 1788810414.37 total 10.00 last 147.97) - open/close 2981 (time 1788810424.37 total 20.00 last 150.10) total: 4500 open/close in 29.79 seconds: 151.05 ops/second - /unlink 3454 (time 1788810446.03 total 10.00 last 345.35) total: 4500 /unlink in 13.68 seconds: 329.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1175 (time 1788810461.73 total 10.01 last 117.43) - open/close 2539 (time 1788810471.73 total 20.01 last 136.39) - open/close 3853 (time 1788810481.73 total 30.01 last 131.33) total: 4500 open/close in 34.97 seconds: 128.70 ops/second - /unlink 2409 (time 1788810499.40 total 10.00 last 240.82) total: 4500 /unlink in 18.82 seconds: 239.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1377 (time 1788810520.69 total 10.01 last 137.58) - open/close 2932 (time 1788810530.69 total 20.01 last 155.46) - open/close 4326 (time 1788810540.69 total 30.01 last 139.40) total: 4500 open/close in 31.09 seconds: 144.74 ops/second - /unlink 2358 (time 1788810553.76 total 10.00 last 235.74) total: 4500 /unlink in 18.80 seconds: 239.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1187 (time 1788810574.66 total 10.00 last 118.65) - open/close 2325 (time 1788810584.66 total 20.00 last 113.79) - open/close 3700 (time 1788810594.66 total 30.01 last 137.48) total: 4500 open/close in 36.83 seconds: 122.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2310 (time 1788810614.13 total 10.00 last 230.93) total: 4500 /unlink in 18.22 seconds: 246.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1232 (time 1788810634.76 total 10.01 last 123.10) - open/close 2512 (time 1788810644.76 total 20.01 last 127.93) - open/close 4026 (time 1788810654.78 total 30.03 last 151.13) total: 4500 open/close in 33.52 seconds: 134.24 ops/second - /unlink 2601 (time 1788810670.19 total 10.00 last 260.03) total: 4500 /unlink in 16.79 seconds: 268.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1200 (time 1788810688.80 total 10.00 last 119.95) - open/close 2564 (time 1788810698.81 total 20.01 last 136.32) - open/close 4338 (time 1788810708.81 total 30.01 last 177.38) total: 4500 open/close in 30.72 seconds: 146.49 ops/second - /unlink 4177 (time 1788810721.27 total 10.00 last 417.67) total: 4500 /unlink in 10.82 seconds: 415.70 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95042 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server unloading modules via unload_modules_local on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing unload_modules_local oleg139-server: modules unloaded.