== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 02:41:05 (1788849665) Stopping clients: oleg631-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg631-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg631-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg631-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg631-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg631-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg631-server oleg631-server: oleg631-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg631-server' oleg631-server: oleg631-server.virtnet: executing load_modules_local oleg631-server: kmemleak disabled oleg631-server: Loading modules from /home/green/git/lustre-release/lustre oleg631-server: detected 4 online CPUs by sysfs oleg631-server: MODOPTS_LIBCFS= oleg631-server: Force libcfs to create 2 CPU partitions oleg631-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg631-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg631-server: oleg631-server.virtnet: executing set_default_debug -1 all pdsh@oleg631-client: oleg631-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg631-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg631-server: oleg631-server.virtnet: executing set_default_debug -1 all pdsh@oleg631-client: oleg631-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg631-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg631-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg631-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg631-server: oleg631-server.virtnet: executing set_default_debug -1 all pdsh@oleg631-client: oleg631-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg631-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg631-client.virtnet: -o user_xattr,flock 192.168.206.131@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 977 (time 1788849814.83 total 10.01 last 97.65) - open/close 2028 (time 1788849824.83 total 20.01 last 105.06) - open/close 3065 (time 1788849834.83 total 30.01 last 103.68) - open/close 4113 (time 1788849844.83 total 40.01 last 104.80) total: 4500 open/close in 43.77 seconds: 102.81 ops/second - /unlink 1844 (time 1788849861.55 total 10.01 last 184.25) - /unlink 3713 (time 1788849871.55 total 20.01 last 186.84) total: 4500 /unlink in 24.42 seconds: 184.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 985 (time 1788849890.07 total 10.00 last 98.47) - open/close 1934 (time 1788849900.08 total 20.01 last 94.83) - open/close 2936 (time 1788849910.08 total 30.01 last 100.16) - open/close 3962 (time 1788849920.09 total 40.02 last 102.50) total: 4500 open/close in 45.01 seconds: 99.98 ops/second - /unlink 2077 (time 1788849937.68 total 10.00 last 207.64) - /unlink 4274 (time 1788849947.69 total 20.01 last 219.52) total: 4500 /unlink in 21.17 seconds: 212.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1044 (time 1788849961.45 total 10.00 last 104.37) - open/close 2086 (time 1788849971.45 total 20.00 last 104.20) - open/close 3130 (time 1788849981.45 total 30.01 last 104.34) - open/close 4169 (time 1788849991.46 total 40.01 last 103.84) total: 4500 open/close in 43.16 seconds: 104.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2151 (time 1788850007.24 total 10.00 last 215.01) - /unlink 4222 (time 1788850017.24 total 20.01 last 207.05) total: 4500 /unlink in 21.53 seconds: 209.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1046 (time 1788850031.67 total 10.01 last 104.53) - open/close 2071 (time 1788850041.68 total 20.01 last 102.45) - open/close 3059 (time 1788850051.68 total 30.01 last 98.78) - open/close 4052 (time 1788850061.69 total 40.03 last 99.19) total: 4500 open/close in 44.79 seconds: 100.46 ops/second - /unlink 2034 (time 1788850079.18 total 10.01 last 203.22) - /unlink 3798 (time 1788850089.19 total 20.01 last 176.30) total: 4500 /unlink in 23.66 seconds: 190.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1001 (time 1788850105.98 total 10.00 last 100.05) - open/close 2045 (time 1788850115.98 total 20.01 last 104.34) - open/close 3107 (time 1788850125.99 total 30.02 last 106.15) - open/close 4128 (time 1788850135.99 total 40.02 last 102.04) total: 4500 open/close in 43.58 seconds: 103.27 ops/second - /unlink 1965 (time 1788850152.26 total 10.00 last 196.44) - /unlink 4013 (time 1788850162.26 total 20.00 last 204.80) total: 4500 /unlink in 22.37 seconds: 201.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1254 (time 1788850177.39 total 10.00 last 125.37) - open/close 2553 (time 1788850187.39 total 20.01 last 129.81) - open/close 3908 (time 1788850197.41 total 30.02 last 135.34) total: 4500 open/close in 35.46 seconds: 126.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2009 (time 1788850215.47 total 10.00 last 200.84) - /unlink 4024 (time 1788850225.47 total 20.00 last 201.46) total: 4500 /unlink in 22.28 seconds: 201.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1197 (time 1788850240.14 total 10.00 last 119.64) - open/close 2354 (time 1788850250.15 total 20.01 last 115.66) - open/close 3533 (time 1788850260.15 total 30.01 last 117.87) total: 4500 open/close in 38.28 seconds: 117.57 ops/second - /unlink 2381 (time 1788850280.90 total 10.00 last 238.04) total: 4500 /unlink in 18.68 seconds: 240.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1226 (time 1788850301.89 total 10.00 last 122.58) - open/close 2316 (time 1788850311.89 total 20.01 last 108.92) - open/close 3288 (time 1788850321.89 total 30.01 last 97.19) - open/close 4337 (time 1788850331.90 total 40.02 last 104.82) total: 4500 open/close in 41.73 seconds: 107.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2117 (time 1788850345.99 total 10.00 last 211.65) - /unlink 4139 (time 1788850356.00 total 20.01 last 202.01) total: 4500 /unlink in 21.79 seconds: 206.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1076 (time 1788850370.36 total 10.00 last 107.58) - open/close 2086 (time 1788850380.36 total 20.00 last 101.00) - open/close 2888 (time 1788850390.37 total 30.01 last 80.14) - open/close 3700 (time 1788850400.37 total 40.01 last 81.19) total: 4500 open/close in 49.89 seconds: 90.19 ops/second - /unlink 2106 (time 1788850422.95 total 10.00 last 210.58) - /unlink 4300 (time 1788850432.95 total 20.00 last 219.39) total: 4500 /unlink in 20.89 seconds: 215.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1228 (time 1788850446.15 total 10.00 last 122.75) - open/close 2641 (time 1788850456.15 total 20.01 last 141.26) - open/close 3860 (time 1788850466.16 total 30.01 last 121.88) total: 4500 open/close in 34.94 seconds: 128.79 ops/second - /unlink 2225 (time 1788850483.34 total 10.00 last 222.47) - /unlink 4454 (time 1788850493.35 total 20.01 last 222.74) total: 4500 /unlink in 20.30 seconds: 221.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1282 (time 1788850506.17 total 10.01 last 128.09) - open/close 2539 (time 1788850516.18 total 20.01 last 125.63) - open/close 3730 (time 1788850526.18 total 30.02 last 119.08) total: 4500 open/close in 36.37 seconds: 123.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2291 (time 1788850544.76 total 10.01 last 228.97) total: 4500 /unlink in 19.47 seconds: 231.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1306 (time 1788850566.44 total 10.00 last 130.57) - open/close 2768 (time 1788850576.44 total 20.00 last 146.20) - open/close 4057 (time 1788850586.44 total 30.00 last 128.90) total: 4500 open/close in 32.90 seconds: 136.77 ops/second - /unlink 3028 (time 1788850601.23 total 10.00 last 302.78) total: 4500 /unlink in 15.07 seconds: 298.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1709 (time 1788850618.23 total 10.00 last 170.86) - open/close 3421 (time 1788850628.23 total 20.00 last 171.19) total: 4500 open/close in 25.70 seconds: 175.08 ops/second - /unlink 3933 (time 1788850645.83 total 10.00 last 393.18) total: 4500 /unlink in 11.54 seconds: 390.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1879 (time 1788850659.33 total 10.00 last 187.89) - open/close 3663 (time 1788850669.33 total 20.00 last 178.33) total: 4500 open/close in 25.37 seconds: 177.40 ops/second - /unlink 3964 (time 1788850686.47 total 10.00 last 396.33) total: 4500 /unlink in 11.41 seconds: 394.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1421 (time 1788850699.72 total 10.01 last 141.98) - open/close 2793 (time 1788850709.72 total 20.02 last 137.10) - open/close 4106 (time 1788850719.73 total 30.02 last 131.26) total: 4500 open/close in 32.77 seconds: 137.33 ops/second - /unlink 3269 (time 1788850734.41 total 10.00 last 326.83) total: 4500 /unlink in 13.36 seconds: 336.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 2021 (time 1788850749.51 total 10.00 last 202.06) - open/close 3749 (time 1788850759.52 total 20.01 last 172.71) total: 4500 open/close in 24.81 seconds: 181.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 3398 (time 1788850776.04 total 10.00 last 339.75) total: 4500 /unlink in 13.76 seconds: 327.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1524 (time 1788850791.60 total 10.00 last 152.40) - open/close 3216 (time 1788850801.60 total 20.00 last 169.20) total: 4500 open/close in 27.31 seconds: 164.79 ops/second - /unlink 3971 (time 1788850820.75 total 10.00 last 397.08) total: 4500 /unlink in 11.53 seconds: 390.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1681 (time 1788850834.18 total 10.00 last 168.02) - open/close 3244 (time 1788850844.18 total 20.01 last 156.26) total: 4500 open/close in 28.47 seconds: 158.03 ops/second - /unlink 2468 (time 1788850865.21 total 10.00 last 246.78) total: 4500 /unlink in 18.71 seconds: 240.47 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95082 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg631-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg631-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg631-server stop mds service on oleg631-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg631-server stop mds service on oleg631-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg631-server unloading modules via unload_modules_local on: 'oleg631-server' oleg631-server: oleg631-server.virtnet: executing unload_modules_local oleg631-server: kmemleak disabled modules unloaded.