== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 02:28:04 (1769758084) Stopping clients: oleg311-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg311-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg311-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg311-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg311-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg311-server oleg311-server: oleg311-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg311-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: MODOPTS_LIBCFS= oleg311-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg311-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg311-server: oleg311-server.virtnet: executing set_default_debug -1 all pdsh@oleg311-client: oleg311-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg311-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg311-server: oleg311-server.virtnet: executing set_default_debug -1 all pdsh@oleg311-client: oleg311-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg311-client.virtnet: -o user_xattr,flock 192.168.203.111@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 902 (time 1769758205.43 total 10.01 last 90.10) - open/close 1875 (time 1769758215.43 total 20.02 last 97.25) - open/close 2826 (time 1769758225.44 total 30.03 last 95.01) - open/close 3709 (time 1769758235.46 total 40.05 last 88.11) total: 4500 open/close in 48.69 seconds: 92.42 ops/second - /unlink 1744 (time 1769758256.84 total 10.01 last 174.31) - /unlink 3500 (time 1769758266.84 total 20.01 last 175.55) total: 4500 /unlink in 25.42 seconds: 177.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 969 (time 1769758285.38 total 10.00 last 96.86) - open/close 1939 (time 1769758295.38 total 20.01 last 96.94) - open/close 2861 (time 1769758305.39 total 30.01 last 92.18) - open/close 3744 (time 1769758315.39 total 40.02 last 88.25) total: 4500 open/close in 49.68 seconds: 90.58 ops/second - /unlink 1765 (time 1769758339.04 total 10.00 last 176.45) - /unlink 3303 (time 1769758349.04 total 20.01 last 153.75) total: 4500 /unlink in 26.04 seconds: 172.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1088 (time 1769758367.73 total 10.01 last 108.69) - open/close 2001 (time 1769758377.75 total 20.02 last 91.18) - open/close 2884 (time 1769758387.75 total 30.03 last 88.29) - open/close 3896 (time 1769758397.75 total 40.03 last 101.15) total: 4500 open/close in 45.55 seconds: 98.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1746 (time 1769758416.28 total 10.00 last 174.52) - /unlink 3527 (time 1769758426.28 total 20.01 last 178.09) total: 4500 /unlink in 25.52 seconds: 176.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1006 (time 1769758444.61 total 10.01 last 100.50) - open/close 2070 (time 1769758454.61 total 20.01 last 106.39) - open/close 3095 (time 1769758464.61 total 30.01 last 102.48) - open/close 4116 (time 1769758474.62 total 40.02 last 102.01) total: 4500 open/close in 43.57 seconds: 103.28 ops/second - /unlink 1842 (time 1769758490.63 total 10.00 last 184.14) - /unlink 3899 (time 1769758500.63 total 20.00 last 205.68) total: 4500 /unlink in 22.65 seconds: 198.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1077 (time 1769758516.02 total 10.00 last 107.69) - open/close 2163 (time 1769758526.03 total 20.01 last 108.51) - open/close 3239 (time 1769758536.04 total 30.01 last 107.57) - open/close 4215 (time 1769758546.04 total 40.01 last 97.59) total: 4500 open/close in 42.92 seconds: 104.84 ops/second - /unlink 2022 (time 1769758561.52 total 10.01 last 202.09) - /unlink 4104 (time 1769758571.52 total 20.01 last 208.17) total: 4500 /unlink in 22.03 seconds: 204.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1076 (time 1769758586.03 total 10.01 last 107.49) - open/close 2184 (time 1769758596.03 total 20.01 last 110.77) - open/close 3308 (time 1769758606.04 total 30.01 last 112.38) - open/close 4436 (time 1769758616.04 total 40.02 last 112.75) total: 4500 open/close in 40.53 seconds: 111.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 2257 (time 1769758629.13 total 10.01 last 225.56) - /unlink 4433 (time 1769758639.14 total 20.01 last 217.57) total: 4500 /unlink in 20.39 seconds: 220.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1227 (time 1769758652.08 total 10.00 last 122.68) - open/close 2613 (time 1769758662.09 total 20.01 last 138.47) - open/close 3960 (time 1769758672.10 total 30.02 last 134.60) total: 4500 open/close in 34.11 seconds: 131.92 ops/second - /unlink 2639 (time 1769758688.31 total 10.00 last 263.85) total: 4500 /unlink in 18.09 seconds: 248.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1225 (time 1769758708.60 total 10.00 last 122.44) - open/close 2567 (time 1769758718.60 total 20.00 last 134.20) - open/close 3874 (time 1769758728.60 total 30.01 last 130.67) total: 4500 open/close in 35.13 seconds: 128.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2458 (time 1769758746.16 total 10.00 last 245.75) total: 4500 /unlink in 18.93 seconds: 237.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1119 (time 1769758768.01 total 10.00 last 111.88) - open/close 2243 (time 1769758778.01 total 20.00 last 112.40) - open/close 3498 (time 1769758788.01 total 30.00 last 125.47) total: 4500 open/close in 36.67 seconds: 122.72 ops/second - /unlink 2280 (time 1769758806.43 total 10.00 last 227.99) total: 4500 /unlink in 19.25 seconds: 233.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1496 (time 1769758827.78 total 10.00 last 149.55) - open/close 2742 (time 1769758837.79 total 20.01 last 124.54) - open/close 4012 (time 1769758847.80 total 30.02 last 126.83) total: 4500 open/close in 33.88 seconds: 132.81 ops/second - /unlink 2150 (time 1769758863.91 total 10.00 last 215.00) - /unlink 4091 (time 1769758873.91 total 20.00 last 194.07) total: 4500 /unlink in 22.08 seconds: 203.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1293 (time 1769758888.38 total 10.00 last 129.25) - open/close 2657 (time 1769758898.39 total 20.01 last 136.39) - open/close 3910 (time 1769758908.39 total 30.01 last 125.20) total: 4500 open/close in 32.93 seconds: 136.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2764 (time 1769758923.30 total 10.00 last 276.33) total: 4500 /unlink in 16.16 seconds: 278.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1597 (time 1769758941.50 total 10.00 last 159.62) - open/close 2997 (time 1769758951.50 total 20.01 last 139.93) - open/close 4263 (time 1769758961.51 total 30.01 last 126.55) total: 4500 open/close in 31.71 seconds: 141.92 ops/second - /unlink 2795 (time 1769758974.89 total 10.00 last 279.43) total: 4500 /unlink in 16.13 seconds: 279.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1302 (time 1769758993.18 total 10.01 last 130.07) - open/close 2623 (time 1769759003.18 total 20.01 last 132.07) - open/close 3847 (time 1769759013.19 total 30.02 last 122.36) total: 4500 open/close in 34.81 seconds: 129.26 ops/second - /unlink 2517 (time 1769759030.30 total 10.00 last 251.64) total: 4500 /unlink in 16.47 seconds: 273.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1381 (time 1769759049.08 total 10.00 last 138.05) - open/close 3102 (time 1769759059.09 total 20.01 last 171.91) total: 4500 open/close in 27.95 seconds: 160.99 ops/second - /unlink 2705 (time 1769759078.94 total 10.01 last 270.31) total: 4500 /unlink in 16.33 seconds: 275.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1500 (time 1769759097.20 total 10.01 last 149.90) - open/close 3242 (time 1769759107.20 total 20.01 last 174.19) total: 4500 open/close in 27.24 seconds: 165.18 ops/second - /unlink 3152 (time 1769759126.28 total 10.00 last 315.20) total: 4500 /unlink in 13.51 seconds: 333.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1465 (time 1769759141.70 total 10.00 last 146.44) - open/close 2907 (time 1769759151.70 total 20.01 last 144.13) - open/close 4294 (time 1769759161.70 total 30.01 last 138.66) total: 4500 open/close in 31.55 seconds: 142.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2978 (time 1769759175.17 total 10.00 last 297.73) total: 4500 /unlink in 15.49 seconds: 290.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1443 (time 1769759192.74 total 10.00 last 144.27) - open/close 2935 (time 1769759202.74 total 20.01 last 149.15) - open/close 4429 (time 1769759212.75 total 30.01 last 149.32) total: 4500 open/close in 30.55 seconds: 147.32 ops/second - /unlink 2792 (time 1769759225.62 total 10.00 last 279.10) total: 4500 /unlink in 14.83 seconds: 303.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1631 (time 1769759242.29 total 10.00 last 163.06) - open/close 3372 (time 1769759252.29 total 20.01 last 174.04) total: 4500 open/close in 27.15 seconds: 165.73 ops/second - /unlink 4291 (time 1769759271.12 total 10.00 last 429.08) total: 4500 /unlink in 10.52 seconds: 427.74 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11101: 74782 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg311-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg311-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg311-server stop mds service on oleg311-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg311-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing unload_modules_local oleg311-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg311-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded.