== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 10:52:15 (1789311135) Stopping clients: oleg351-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg351-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg351-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg351-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg351-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg351-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg351-server oleg351-server: oleg351-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg351-server' oleg351-server: oleg351-server.virtnet: executing load_modules_local oleg351-server: kmemleak disabled oleg351-server: Loading modules from /home/green/git/lustre-release/lustre oleg351-server: detected 4 online CPUs by sysfs oleg351-server: MODOPTS_LIBCFS= oleg351-server: Force libcfs to create 2 CPU partitions oleg351-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg351-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg351-server: oleg351-server.virtnet: executing set_default_debug -1 all pdsh@oleg351-client: oleg351-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg351-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg351-server: oleg351-server.virtnet: executing set_default_debug -1 all pdsh@oleg351-client: oleg351-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg351-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg351-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg351-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg351-server: oleg351-server.virtnet: executing set_default_debug -1 all pdsh@oleg351-client: oleg351-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg351-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg351-client.virtnet: -o user_xattr,flock 192.168.203.151@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 832 (time 1789311339.10 total 10.01 last 83.13) - open/close 1570 (time 1789311349.10 total 20.01 last 73.80) - open/close 2303 (time 1789311359.10 total 30.01 last 73.29) - open/close 3016 (time 1789311369.11 total 40.02 last 71.27) - open/close 3808 (time 1789311379.12 total 50.03 last 79.11) total: 4500 open/close in 59.49 seconds: 75.64 ops/second - /unlink 1600 (time 1789311402.53 total 10.00 last 160.00) - /unlink 3000 (time 1789311412.53 total 20.00 last 139.96) - /unlink 4405 (time 1789311422.54 total 30.01 last 140.41) total: 4500 /unlink in 30.68 seconds: 146.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 813 (time 1789311436.98 total 10.02 last 81.17) - open/close 1624 (time 1789311446.98 total 20.02 last 81.09) - open/close 2363 (time 1789311456.99 total 30.03 last 73.82) - open/close 3097 (time 1789311466.99 total 40.03 last 73.37) - open/close 3877 (time 1789311476.99 total 50.03 last 78.00) total: 4500 open/close in 57.46 seconds: 78.31 ops/second - /unlink 1679 (time 1789311499.04 total 10.01 last 167.81) - /unlink 3295 (time 1789311509.04 total 20.01 last 161.58) total: 4500 /unlink in 27.46 seconds: 163.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 842 (time 1789311531.02 total 10.00 last 84.17) - open/close 1686 (time 1789311541.03 total 20.01 last 84.36) - open/close 2504 (time 1789311551.04 total 30.02 last 81.71) - open/close 3369 (time 1789311561.05 total 40.03 last 86.45) - open/close 4257 (time 1789311571.05 total 50.03 last 88.78) total: 4500 open/close in 52.86 seconds: 85.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1697 (time 1789311587.92 total 10.01 last 169.58) - /unlink 2961 (time 1789311597.92 total 20.01 last 126.36) - /unlink 4346 (time 1789311607.92 total 30.01 last 138.49) total: 4500 /unlink in 31.40 seconds: 143.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 699 (time 1789311624.42 total 10.01 last 69.84) - open/close 1478 (time 1789311634.45 total 20.04 last 77.69) - open/close 2157 (time 1789311644.47 total 30.05 last 67.78) - open/close 2868 (time 1789311654.47 total 40.05 last 71.09) - open/close 3600 (time 1789311664.49 total 50.08 last 73.04) - open/close 4327 (time 1789311674.50 total 60.08 last 72.65) total: 4500 open/close in 61.92 seconds: 72.67 ops/second - /unlink 1650 (time 1789311690.90 total 10.00 last 165.00) - /unlink 3212 (time 1789311700.90 total 20.00 last 156.15) total: 4500 /unlink in 28.49 seconds: 157.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 749 (time 1789311724.63 total 10.00 last 74.89) - open/close 1519 (time 1789311734.64 total 20.01 last 76.93) - open/close 2350 (time 1789311744.64 total 30.01 last 83.10) - open/close 3068 (time 1789311754.66 total 40.03 last 71.67) - open/close 3820 (time 1789311764.66 total 50.03 last 75.17) total: 4500 open/close in 59.22 seconds: 75.99 ops/second - /unlink 1459 (time 1789311788.06 total 10.00 last 145.85) - /unlink 2873 (time 1789311798.06 total 20.01 last 141.36) - /unlink 4431 (time 1789311808.07 total 30.02 last 155.65) total: 4500 /unlink in 30.81 seconds: 146.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 537 (time 1789311824.31 total 10.02 last 53.59) - open/close 1153 (time 1789311834.32 total 20.02 last 61.59) - open/close 1941 (time 1789311844.34 total 30.04 last 78.63) - open/close 2563 (time 1789311854.34 total 40.04 last 62.20) - open/close 3380 (time 1789311864.34 total 50.05 last 81.69) - open/close 4174 (time 1789311874.34 total 60.05 last 79.39) total: 4500 open/close in 63.85 seconds: 70.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1532 (time 1789311892.57 total 10.00 last 153.16) - /unlink 3006 (time 1789311902.58 total 20.01 last 147.24) total: 4500 /unlink in 29.76 seconds: 151.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 861 (time 1789311926.64 total 10.00 last 86.06) - open/close 1688 (time 1789311936.66 total 20.02 last 82.59) - open/close 2538 (time 1789311946.66 total 30.02 last 84.95) - open/close 3317 (time 1789311956.67 total 40.03 last 77.87) - open/close 4083 (time 1789311966.67 total 50.03 last 76.55) total: 4500 open/close in 56.53 seconds: 79.60 ops/second - /unlink 1417 (time 1789311988.85 total 10.01 last 141.57) - /unlink 2986 (time 1789311998.85 total 20.01 last 156.82) - /unlink 4449 (time 1789312008.86 total 30.02 last 146.16) total: 4500 /unlink in 30.56 seconds: 147.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 785 (time 1789312023.43 total 10.00 last 78.49) - open/close 1615 (time 1789312033.44 total 20.01 last 82.93) - open/close 2489 (time 1789312043.44 total 30.01 last 87.36) - open/close 3249 (time 1789312053.45 total 40.02 last 75.91) - open/close 4092 (time 1789312063.47 total 50.04 last 84.16) total: 4500 open/close in 55.00 seconds: 81.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1699 (time 1789312082.50 total 10.00 last 169.87) - /unlink 3205 (time 1789312092.50 total 20.00 last 150.57) total: 4500 /unlink in 27.98 seconds: 160.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 785 (time 1789312114.97 total 10.00 last 78.47) - open/close 1602 (time 1789312124.98 total 20.01 last 81.63) - open/close 2375 (time 1789312134.99 total 30.01 last 77.28) - open/close 3213 (time 1789312145.01 total 40.03 last 83.64) - open/close 3966 (time 1789312155.01 total 50.04 last 75.24) total: 4500 open/close in 56.79 seconds: 79.24 ops/second - /unlink 1596 (time 1789312175.45 total 10.00 last 159.54) - /unlink 3077 (time 1789312185.46 total 20.01 last 147.99) total: 4500 /unlink in 29.76 seconds: 151.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 840 (time 1789312209.59 total 10.01 last 83.91) - open/close 1611 (time 1789312219.60 total 20.02 last 77.04) - open/close 2367 (time 1789312229.61 total 30.03 last 75.53) - open/close 3155 (time 1789312239.63 total 40.05 last 78.62) - open/close 4015 (time 1789312249.65 total 50.06 last 85.88) total: 4500 open/close in 55.62 seconds: 80.90 ops/second - /unlink 1566 (time 1789312268.59 total 10.00 last 156.57) - /unlink 3054 (time 1789312278.59 total 20.00 last 148.78) total: 4500 /unlink in 28.73 seconds: 156.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 835 (time 1789312302.15 total 10.01 last 83.44) - open/close 1676 (time 1789312312.16 total 20.02 last 84.02) - open/close 2485 (time 1789312322.17 total 30.02 last 80.85) - open/close 3343 (time 1789312332.17 total 40.03 last 85.76) - open/close 4195 (time 1789312342.17 total 50.03 last 85.17) total: 4500 open/close in 53.37 seconds: 84.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1399 (time 1789312359.73 total 10.00 last 139.90) - /unlink 2917 (time 1789312369.74 total 20.01 last 151.70) - /unlink 4446 (time 1789312379.74 total 30.01 last 152.83) total: 4500 /unlink in 30.60 seconds: 147.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 832 (time 1789312394.32 total 10.01 last 83.14) - open/close 1683 (time 1789312404.32 total 20.01 last 85.07) - open/close 2530 (time 1789312414.33 total 30.02 last 84.63) - open/close 3191 (time 1789312424.34 total 40.03 last 66.04) - open/close 3989 (time 1789312434.36 total 50.05 last 79.63) total: 4500 open/close in 55.91 seconds: 80.49 ops/second - /unlink 1609 (time 1789312453.92 total 10.00 last 160.83) - /unlink 3115 (time 1789312463.93 total 20.01 last 150.52) total: 4500 /unlink in 28.81 seconds: 156.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 833 (time 1789312486.65 total 10.00 last 83.27) - open/close 1641 (time 1789312496.65 total 20.00 last 80.80) - open/close 2474 (time 1789312506.65 total 30.01 last 83.28) - open/close 3351 (time 1789312516.66 total 40.01 last 87.64) - open/close 4159 (time 1789312526.66 total 50.01 last 80.79) total: 4500 open/close in 54.71 seconds: 82.25 ops/second - /unlink 1571 (time 1789312544.92 total 10.01 last 156.93) - /unlink 3116 (time 1789312554.92 total 20.01 last 154.48) total: 4500 /unlink in 29.51 seconds: 152.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 840 (time 1789312578.52 total 10.00 last 83.98) - open/close 1745 (time 1789312588.52 total 20.01 last 90.47) - open/close 2608 (time 1789312598.54 total 30.03 last 86.11) - open/close 3484 (time 1789312608.54 total 40.03 last 87.58) - open/close 4301 (time 1789312618.55 total 50.03 last 81.65) total: 4500 open/close in 52.53 seconds: 85.67 ops/second - /unlink 1691 (time 1789312634.30 total 10.00 last 169.03) - /unlink 3049 (time 1789312644.30 total 20.01 last 135.75) total: 4500 /unlink in 28.90 seconds: 155.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 762 (time 1789312667.16 total 10.01 last 76.15) - open/close 1570 (time 1789312677.17 total 20.02 last 80.71) - open/close 2442 (time 1789312687.18 total 30.03 last 87.09) - open/close 3270 (time 1789312697.18 total 40.03 last 82.77) - open/close 4132 (time 1789312707.19 total 50.04 last 86.18) total: 4500 open/close in 54.62 seconds: 82.38 ops/second - /unlink 1640 (time 1789312725.75 total 10.00 last 163.97) - /unlink 3226 (time 1789312735.76 total 20.01 last 158.52) total: 4500 /unlink in 27.97 seconds: 160.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 874 (time 1789312757.02 total 10.00 last 87.36) - open/close 1787 (time 1789312767.03 total 20.02 last 91.19) - open/close 2733 (time 1789312777.04 total 30.02 last 94.59) - open/close 3684 (time 1789312787.05 total 40.03 last 94.96) total: 4500 open/close in 47.87 seconds: 94.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1887 (time 1789312807.47 total 10.00 last 188.68) - /unlink 3661 (time 1789312817.48 total 20.01 last 177.21) total: 4500 /unlink in 24.63 seconds: 182.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 959 (time 1789312834.94 total 10.01 last 95.80) - open/close 1921 (time 1789312844.94 total 20.01 last 96.17) - open/close 2911 (time 1789312854.95 total 30.02 last 98.92) - open/close 3821 (time 1789312864.95 total 40.02 last 90.98) total: 4500 open/close in 47.28 seconds: 95.18 ops/second - /unlink 1909 (time 1789312885.30 total 10.01 last 190.73) - /unlink 3764 (time 1789312895.30 total 20.01 last 185.46) total: 4500 /unlink in 23.68 seconds: 190.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 951 (time 1789312912.10 total 10.00 last 95.08) - open/close 1899 (time 1789312922.10 total 20.01 last 94.76) - open/close 2793 (time 1789312932.11 total 30.01 last 89.32) - open/close 3781 (time 1789312942.12 total 40.02 last 98.72) total: 4500 open/close in 47.34 seconds: 95.05 ops/second - /unlink 1865 (time 1789312962.29 total 10.00 last 186.47) - /unlink 3669 (time 1789312972.30 total 20.00 last 180.34) total: 4500 /unlink in 24.47 seconds: 183.86 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 94882 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg351-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg351-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg351-server stop mds service on oleg351-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg351-server stop mds service on oleg351-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg351-server unloading modules via unload_modules_local on: 'oleg351-server' oleg351-server: oleg351-server.virtnet: executing unload_modules_local oleg351-server: kmemleak disabled modules unloaded.