== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 11:04:06 (1780326246) Stopping clients: oleg215-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg215-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg215-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg215-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg215-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg215-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg215-server oleg215-server: oleg215-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg215-server' oleg215-server: oleg215-server.virtnet: executing load_modules_local oleg215-server: Loading modules from /home/green/git/lustre-release/lustre oleg215-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg215-server: detected 4 online CPUs by sysfs oleg215-server: MODOPTS_LIBCFS= oleg215-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg215-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg215-server: oleg215-server.virtnet: executing set_default_debug -1 all pdsh@oleg215-client: oleg215-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg215-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg215-server: oleg215-server.virtnet: executing set_default_debug -1 all pdsh@oleg215-client: oleg215-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg215-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg215-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg215-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg215-server: oleg215-server.virtnet: executing set_default_debug -1 all pdsh@oleg215-client: oleg215-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg215-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg215-client.virtnet: -o user_xattr,flock 192.168.202.115@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1362 (time 1780326365.65 total 10.00 last 136.14) - open/close 2807 (time 1780326375.65 total 20.01 last 144.45) - open/close 4202 (time 1780326385.66 total 30.01 last 139.45) total: 4500 open/close in 32.05 seconds: 140.40 ops/second - /unlink 2624 (time 1780326400.07 total 10.00 last 262.33) total: 4500 /unlink in 16.25 seconds: 276.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1376 (time 1780326418.47 total 10.01 last 137.50) - open/close 2763 (time 1780326428.47 total 20.01 last 138.69) - open/close 4058 (time 1780326438.47 total 30.01 last 129.50) total: 4500 open/close in 33.16 seconds: 135.70 ops/second - /unlink 2784 (time 1780326453.61 total 10.00 last 278.39) total: 4500 /unlink in 16.23 seconds: 277.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1853 (time 1780326472.29 total 10.00 last 185.29) - open/close 3485 (time 1780326482.29 total 20.01 last 163.12) total: 4500 open/close in 26.31 seconds: 171.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2840 (time 1780326500.51 total 10.00 last 284.00) total: 4500 /unlink in 15.80 seconds: 284.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1877 (time 1780326517.87 total 10.01 last 187.57) - open/close 3528 (time 1780326527.88 total 20.01 last 165.02) total: 4500 open/close in 26.32 seconds: 170.96 ops/second - /unlink 3280 (time 1780326546.12 total 10.00 last 327.92) total: 4500 /unlink in 13.89 seconds: 323.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1564 (time 1780326561.74 total 10.01 last 156.30) - open/close 3354 (time 1780326571.74 total 20.01 last 178.96) total: 4500 open/close in 25.85 seconds: 174.06 ops/second - /unlink 3638 (time 1780326589.31 total 10.00 last 363.74) total: 4500 /unlink in 12.19 seconds: 369.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1958 (time 1780326603.30 total 10.00 last 195.72) - open/close 3732 (time 1780326613.30 total 20.01 last 177.38) total: 4500 open/close in 25.01 seconds: 179.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2808 (time 1780326630.44 total 10.00 last 280.74) total: 4500 /unlink in 15.82 seconds: 284.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1967 (time 1780326648.09 total 10.00 last 196.68) - open/close 3423 (time 1780326658.09 total 20.00 last 145.59) total: 4500 open/close in 27.62 seconds: 162.95 ops/second - /unlink 3409 (time 1780326677.75 total 10.00 last 340.87) total: 4500 /unlink in 13.32 seconds: 337.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1749 (time 1780326692.74 total 10.01 last 174.79) - open/close 3236 (time 1780326702.74 total 20.01 last 148.66) total: 4500 open/close in 26.76 seconds: 168.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 3257 (time 1780326721.34 total 10.00 last 325.66) total: 4500 /unlink in 13.24 seconds: 339.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1792 (time 1780326736.41 total 10.01 last 179.10) - open/close 3315 (time 1780326746.42 total 20.01 last 152.16) total: 4500 open/close in 27.55 seconds: 163.36 ops/second - /unlink 1881 (time 1780326766.65 total 10.00 last 188.01) - /unlink 3729 (time 1780326776.65 total 20.00 last 184.80) total: 4500 /unlink in 24.46 seconds: 183.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 965 (time 1780326796.32 total 10.00 last 96.45) - open/close 1751 (time 1780326806.33 total 20.01 last 78.55) - open/close 2394 (time 1780326816.34 total 30.02 last 64.25) - open/close 3087 (time 1780326826.35 total 40.03 last 69.20) - open/close 3991 (time 1780326836.36 total 50.04 last 90.32) total: 4500 open/close in 56.39 seconds: 79.80 ops/second - /unlink 1611 (time 1780326857.47 total 10.00 last 161.07) - /unlink 3244 (time 1780326867.47 total 20.01 last 163.25) total: 4500 /unlink in 27.65 seconds: 162.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 954 (time 1780326889.47 total 10.01 last 95.31) - open/close 1952 (time 1780326899.47 total 20.01 last 99.80) - open/close 2953 (time 1780326909.47 total 30.01 last 100.07) - open/close 3888 (time 1780326919.49 total 40.03 last 93.36) total: 4500 open/close in 46.41 seconds: 96.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1908 (time 1780326938.92 total 10.01 last 190.66) - /unlink 3773 (time 1780326948.92 total 20.01 last 186.44) total: 4500 /unlink in 24.19 seconds: 186.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 976 (time 1780326966.73 total 10.03 last 97.34) - open/close 1973 (time 1780326976.74 total 20.03 last 99.62) - open/close 2942 (time 1780326986.74 total 30.04 last 96.84) - open/close 3927 (time 1780326996.75 total 40.05 last 98.44) total: 4500 open/close in 46.22 seconds: 97.36 ops/second - /unlink 1708 (time 1780327015.89 total 10.00 last 170.74) - /unlink 3412 (time 1780327025.90 total 20.01 last 170.36) total: 4500 /unlink in 26.26 seconds: 171.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 945 (time 1780327046.46 total 10.00 last 94.46) - open/close 1850 (time 1780327056.47 total 20.01 last 90.47) - open/close 2637 (time 1780327066.47 total 30.01 last 78.66) - open/close 3482 (time 1780327076.48 total 40.02 last 84.47) - open/close 4372 (time 1780327086.49 total 50.03 last 88.88) total: 4500 open/close in 51.47 seconds: 87.43 ops/second - /unlink 1763 (time 1780327101.03 total 10.00 last 176.24) - /unlink 3209 (time 1780327111.03 total 20.01 last 144.56) total: 4500 /unlink in 27.88 seconds: 161.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 978 (time 1780327132.98 total 10.00 last 97.76) - open/close 1924 (time 1780327142.99 total 20.01 last 94.53) - open/close 2982 (time 1780327152.99 total 30.02 last 105.75) - open/close 3920 (time 1780327163.01 total 40.04 last 93.60) total: 4500 open/close in 46.53 seconds: 96.71 ops/second - /unlink 1737 (time 1780327182.93 total 10.00 last 173.70) - /unlink 3464 (time 1780327192.94 total 20.00 last 172.63) total: 4500 /unlink in 26.17 seconds: 171.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 921 (time 1780327212.43 total 10.01 last 92.00) - open/close 1826 (time 1780327222.45 total 20.03 last 90.35) - open/close 2731 (time 1780327232.46 total 30.04 last 90.40) - open/close 3635 (time 1780327242.47 total 40.05 last 90.33) total: 4500 open/close in 48.92 seconds: 91.98 ops/second - /unlink 1940 (time 1780327264.97 total 10.01 last 193.90) - /unlink 3594 (time 1780327274.98 total 20.01 last 165.35) total: 4500 /unlink in 26.02 seconds: 172.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 971 (time 1780327294.57 total 10.00 last 97.10) - open/close 1978 (time 1780327304.57 total 20.00 last 100.70) - open/close 3077 (time 1780327314.58 total 30.01 last 109.82) - open/close 4097 (time 1780327324.60 total 40.02 last 101.82) total: 4500 open/close in 43.86 seconds: 102.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1890 (time 1780327342.02 total 10.00 last 188.98) - /unlink 3829 (time 1780327352.02 total 20.00 last 193.90) total: 4500 /unlink in 23.84 seconds: 188.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 960 (time 1780327369.75 total 10.00 last 95.96) - open/close 1971 (time 1780327379.76 total 20.01 last 101.03) - open/close 2975 (time 1780327389.78 total 30.03 last 100.22) - open/close 3927 (time 1780327399.79 total 40.04 last 95.08) total: 4500 open/close in 45.86 seconds: 98.12 ops/second - /unlink 1947 (time 1780327418.81 total 10.00 last 194.63) - /unlink 3823 (time 1780327428.81 total 20.01 last 187.57) total: 4500 /unlink in 23.29 seconds: 193.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 987 (time 1780327445.05 total 10.00 last 98.67) - open/close 2039 (time 1780327455.05 total 20.01 last 105.11) - open/close 3031 (time 1780327465.05 total 30.01 last 99.19) - open/close 4105 (time 1780327475.07 total 40.02 last 107.27) total: 4500 open/close in 43.54 seconds: 103.34 ops/second - /unlink 2198 (time 1780327491.43 total 10.00 last 219.80) total: 4500 /unlink in 19.52 seconds: 230.54 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11196: 93382 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg215-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg215-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg215-server stop mds service on oleg215-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg215-server stop mds service on oleg215-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg215-server unloading modules via unload_modules_local on: 'oleg215-server' oleg215-server: oleg215-server.virtnet: executing unload_modules_local oleg215-server: modules unloaded.