== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 08:58:17 (1788267497) Stopping clients: oleg250-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg250-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg250-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg250-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg250-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg250-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg250-server oleg250-server: oleg250-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg250-server' oleg250-server: oleg250-server.virtnet: executing load_modules_local oleg250-server: kmemleak disabled oleg250-server: Loading modules from /home/green/git/lustre-release/lustre oleg250-server: detected 4 online CPUs by sysfs oleg250-server: MODOPTS_LIBCFS= oleg250-server: Force libcfs to create 2 CPU partitions oleg250-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg250-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg250-server: oleg250-server.virtnet: executing set_default_debug -1 all pdsh@oleg250-client: oleg250-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg250-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg250-server: oleg250-server.virtnet: executing set_default_debug -1 all pdsh@oleg250-client: oleg250-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg250-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg250-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg250-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg250-server: oleg250-server.virtnet: executing set_default_debug -1 all pdsh@oleg250-client: oleg250-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg250-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg250-client.virtnet: -o user_xattr,flock 192.168.202.150@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 881 (time 1788267647.88 total 10.01 last 88.03) - open/close 1768 (time 1788267657.89 total 20.01 last 88.66) - open/close 2645 (time 1788267667.89 total 30.01 last 87.69) - open/close 3589 (time 1788267677.91 total 40.03 last 94.22) total: 4500 open/close in 49.99 seconds: 90.01 ops/second - /unlink 1897 (time 1788267701.04 total 10.00 last 189.69) - /unlink 3986 (time 1788267711.04 total 20.00 last 208.84) total: 4500 /unlink in 22.35 seconds: 201.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 957 (time 1788267726.64 total 10.01 last 95.64) - open/close 1895 (time 1788267736.64 total 20.01 last 93.76) - open/close 2871 (time 1788267746.64 total 30.01 last 97.58) - open/close 3850 (time 1788267756.65 total 40.02 last 97.85) total: 4500 open/close in 46.90 seconds: 95.95 ops/second - /unlink 1950 (time 1788267776.21 total 10.00 last 194.99) - /unlink 3925 (time 1788267786.22 total 20.00 last 197.45) total: 4500 /unlink in 22.88 seconds: 196.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1064 (time 1788267801.90 total 10.00 last 106.38) - open/close 2170 (time 1788267811.91 total 20.00 last 110.57) - open/close 3434 (time 1788267821.91 total 30.01 last 126.35) - open/close 4471 (time 1788267831.92 total 40.02 last 103.62) total: 4500 open/close in 40.31 seconds: 111.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1956 (time 1788267844.97 total 10.00 last 195.56) - /unlink 3926 (time 1788267854.98 total 20.01 last 196.85) total: 4500 /unlink in 23.22 seconds: 193.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1086 (time 1788267870.67 total 10.00 last 108.60) - open/close 2116 (time 1788267880.67 total 20.00 last 103.00) - open/close 3252 (time 1788267890.67 total 30.00 last 113.58) - open/close 4233 (time 1788267900.68 total 40.01 last 97.99) total: 4500 open/close in 42.57 seconds: 105.71 ops/second - /unlink 2039 (time 1788267915.74 total 10.00 last 203.86) - /unlink 4301 (time 1788267925.74 total 20.00 last 226.15) total: 4500 /unlink in 20.84 seconds: 215.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1042 (time 1788267939.97 total 10.01 last 104.14) - open/close 2042 (time 1788267949.97 total 20.01 last 99.93) - open/close 3035 (time 1788267959.97 total 30.02 last 99.27) - open/close 3944 (time 1788267969.99 total 40.03 last 90.77) total: 4500 open/close in 46.16 seconds: 97.48 ops/second - /unlink 2344 (time 1788267988.84 total 10.00 last 234.35) - /unlink 4322 (time 1788267998.84 total 20.00 last 197.77) total: 4500 /unlink in 20.87 seconds: 215.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1147 (time 1788268012.26 total 10.01 last 114.64) - open/close 2410 (time 1788268022.26 total 20.01 last 126.29) - open/close 3626 (time 1788268032.27 total 30.01 last 121.54) total: 4500 open/close in 37.96 seconds: 118.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2467 (time 1788268052.64 total 10.00 last 246.66) - /unlink 4379 (time 1788268062.64 total 20.00 last 191.14) total: 4500 /unlink in 20.55 seconds: 218.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 937 (time 1788268075.69 total 10.00 last 93.69) - open/close 2038 (time 1788268085.69 total 20.00 last 110.06) - open/close 3076 (time 1788268095.69 total 30.01 last 103.78) - open/close 4208 (time 1788268105.70 total 40.01 last 113.19) total: 4500 open/close in 42.59 seconds: 105.67 ops/second - /unlink 2120 (time 1788268120.57 total 10.00 last 211.93) - /unlink 4288 (time 1788268130.57 total 20.00 last 216.78) total: 4500 /unlink in 21.09 seconds: 213.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1091 (time 1788268144.53 total 10.01 last 109.01) - open/close 2110 (time 1788268154.54 total 20.01 last 101.88) - open/close 3148 (time 1788268164.55 total 30.02 last 103.69) - open/close 4208 (time 1788268174.56 total 40.04 last 105.84) total: 4500 open/close in 42.84 seconds: 105.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2496 (time 1788268189.68 total 10.00 last 249.57) total: 4500 /unlink in 18.50 seconds: 243.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1172 (time 1788268210.37 total 10.01 last 117.13) - open/close 2199 (time 1788268220.37 total 20.01 last 102.65) - open/close 3215 (time 1788268230.37 total 30.01 last 101.56) - open/close 4394 (time 1788268240.38 total 40.02 last 117.85) total: 4500 open/close in 40.82 seconds: 110.25 ops/second - /unlink 2309 (time 1788268253.45 total 10.00 last 230.86) total: 4500 /unlink in 19.42 seconds: 231.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1081 (time 1788268275.28 total 10.01 last 107.96) - open/close 2228 (time 1788268285.28 total 20.01 last 114.68) - open/close 3388 (time 1788268295.28 total 30.02 last 115.93) total: 4500 open/close in 39.86 seconds: 112.89 ops/second - /unlink 2184 (time 1788268317.42 total 10.00 last 218.31) - /unlink 4280 (time 1788268327.42 total 20.01 last 209.51) total: 4500 /unlink in 21.38 seconds: 210.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1167 (time 1788268341.13 total 10.00 last 116.65) - open/close 2283 (time 1788268351.14 total 20.01 last 111.54) - open/close 3360 (time 1788268361.14 total 30.01 last 107.65) total: 4500 open/close in 37.67 seconds: 119.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2345 (time 1788268380.86 total 10.00 last 234.48) total: 4500 /unlink in 18.02 seconds: 249.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1468 (time 1788268400.87 total 10.00 last 146.74) - open/close 2687 (time 1788268410.88 total 20.01 last 121.83) - open/close 4295 (time 1788268420.88 total 30.01 last 160.75) total: 4500 open/close in 31.48 seconds: 142.97 ops/second - /unlink 3378 (time 1788268434.50 total 10.00 last 337.78) total: 4500 /unlink in 13.37 seconds: 336.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1659 (time 1788268449.76 total 10.00 last 165.85) - open/close 3099 (time 1788268459.76 total 20.01 last 143.90) - open/close 4354 (time 1788268469.76 total 30.01 last 125.49) total: 4500 open/close in 31.15 seconds: 144.44 ops/second - /unlink 3033 (time 1788268482.89 total 10.00 last 303.30) total: 4500 /unlink in 15.97 seconds: 281.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1569 (time 1788268501.12 total 10.01 last 156.75) - open/close 2955 (time 1788268511.12 total 20.01 last 138.59) - open/close 4338 (time 1788268521.12 total 30.01 last 138.28) total: 4500 open/close in 31.16 seconds: 144.40 ops/second - /unlink 2818 (time 1788268534.38 total 10.00 last 281.76) total: 4500 /unlink in 16.02 seconds: 280.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1429 (time 1788268552.50 total 10.01 last 142.82) - open/close 3292 (time 1788268562.50 total 20.01 last 186.27) total: 4500 open/close in 26.18 seconds: 171.86 ops/second - /unlink 3505 (time 1788268580.51 total 10.00 last 350.42) total: 4500 /unlink in 12.57 seconds: 357.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1802 (time 1788268594.80 total 10.01 last 180.05) - open/close 3526 (time 1788268604.80 total 20.01 last 172.37) total: 4500 open/close in 25.68 seconds: 175.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2441 (time 1788268622.27 total 10.00 last 244.10) total: 4500 /unlink in 18.39 seconds: 244.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1564 (time 1788268642.64 total 10.01 last 156.31) - open/close 2642 (time 1788268652.64 total 20.01 last 107.79) - open/close 3884 (time 1788268662.65 total 30.01 last 124.09) total: 4500 open/close in 34.99 seconds: 128.63 ops/second - /unlink 2687 (time 1788268679.55 total 10.00 last 268.68) total: 4500 /unlink in 16.39 seconds: 274.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1541 (time 1788268698.04 total 10.00 last 154.05) - open/close 2854 (time 1788268708.04 total 20.00 last 131.28) - open/close 3985 (time 1788268718.05 total 30.01 last 113.01) total: 4500 open/close in 34.80 seconds: 129.32 ops/second - /unlink 2058 (time 1788268735.11 total 10.00 last 205.79) - /unlink 4400 (time 1788268745.11 total 20.00 last 234.14) total: 4500 /unlink in 20.48 seconds: 219.73 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95216 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg250-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg250-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg250-server stop mds service on oleg250-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg250-server stop mds service on oleg250-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg250-server unloading modules via unload_modules_local on: 'oleg250-server' oleg250-server: oleg250-server.virtnet: executing unload_modules_local oleg250-server: kmemleak disabled modules unloaded.