[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 456611245 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003111] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008323] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.011007] pid_max: default: 32768 minimum: 301 [ 0.012136] LSM: Security Framework initializing [ 0.013056] Yama: becoming mindful. [ 0.014033] SELinux: Initializing. [ 0.015071] *** VALIDATE selinux *** [ 0.023489] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027293] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028152] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030114] *** VALIDATE tmpfs *** [ 0.032335] *** VALIDATE proc *** [ 0.033232] *** VALIDATE cgroup *** [ 0.034024] *** VALIDATE cgroup2 *** [ 0.035281] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037077] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039025] Spectre V2 : User space: Vulnerable [ 0.040006] Speculative Store Bypass: Vulnerable [ 0.042950] debug: unmapping init [mem 0xffffffffba259000-0xffffffffba260fff] [ 0.044172] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045689] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046020] ... version: 2 [ 0.047011] ... bit width: 48 [ 0.048009] ... generic registers: 4 [ 0.049011] ... value mask: 0000ffffffffffff [ 0.050014] ... max period: 00007fffffffffff [ 0.051016] ... fixed-purpose events: 3 [ 0.052013] ... event mask: 000000070000000f [ 0.053305] rcu: Hierarchical SRCU implementation. [ 0.055346] smp: Bringing up secondary CPUs ... [ 0.056516] x86: Booting SMP configuration: [ 0.057016] .... node #0, CPUs: #1 #2 #3 [ 0.060289] smp: Brought up 1 node, 4 CPUs [ 0.062010] smpboot: Max logical packages: 1 [ 0.063012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.126015] node 0 deferred pages initialised in 60ms [ 0.128144] devtmpfs: initialized [ 0.129187] x86/mm: Memory block size: 128MB [ 0.131131] gcov: version magic: 0x41383552 [ 0.133186] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.134049] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.135209] pinctrl core: initialized pinctrl subsystem [ 0.136143] [ 0.136426] ************************************************************* [ 0.137007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.138008] ** ** [ 0.139007] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.140007] ** ** [ 0.141007] ** This means that this kernel is built to expose internal ** [ 0.142009] ** IOMMU data structures, which may compromise security on ** [ 0.143006] ** your system. ** [ 0.144010] ** ** [ 0.145011] ** If you see this message and you are not debugging the ** [ 0.146009] ** kernel, report this immediately to your vendor! ** [ 0.147009] ** ** [ 0.148008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.149007] ************************************************************* [ 0.150583] NET: Registered protocol family 16 [ 0.151331] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.152035] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.153047] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.154409] cpuidle: using governor menu [ 0.155892] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.158453] PCI: Using configuration type 1 for base access [ 0.159428] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.169055] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.170020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.171198] cryptd: max_cpu_qlen set to 1000 [ 0.172264] ACPI: Added _OSI(Module Device) [ 0.173016] ACPI: Added _OSI(Processor Device) [ 0.174013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.175013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.179259] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.183055] ACPI: Interpreter enabled [ 0.184052] ACPI: PM: (supports S0 S3 S4 S5) [ 0.185014] ACPI: Using IOAPIC for interrupt routing [ 0.186120] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.187398] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.196654] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.197031] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.198017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.199073] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.201385] acpiphp: Slot [2] registered [ 0.202104] acpiphp: Slot [5] registered [ 0.203118] acpiphp: Slot [6] registered [ 0.204078] acpiphp: Slot [7] registered [ 0.204885] acpiphp: Slot [8] registered [ 0.205072] acpiphp: Slot [9] registered [ 0.205880] acpiphp: Slot [10] registered [ 0.206072] acpiphp: Slot [3] registered [ 0.206927] acpiphp: Slot [4] registered [ 0.207054] acpiphp: Slot [11] registered [ 0.207833] acpiphp: Slot [12] registered [ 0.208062] acpiphp: Slot [13] registered [ 0.208871] acpiphp: Slot [14] registered [ 0.209046] acpiphp: Slot [15] registered [ 0.209859] acpiphp: Slot [16] registered [ 0.210069] acpiphp: Slot [17] registered [ 0.210848] acpiphp: Slot [18] registered [ 0.211043] acpiphp: Slot [19] registered [ 0.211757] acpiphp: Slot [20] registered [ 0.212059] acpiphp: Slot [21] registered [ 0.213090] acpiphp: Slot [22] registered [ 0.214065] acpiphp: Slot [23] registered [ 0.215100] acpiphp: Slot [24] registered [ 0.216064] acpiphp: Slot [25] registered [ 0.216991] acpiphp: Slot [26] registered [ 0.217053] acpiphp: Slot [27] registered [ 0.217901] acpiphp: Slot [28] registered [ 0.218063] acpiphp: Slot [29] registered [ 0.218969] acpiphp: Slot [30] registered [ 0.219064] acpiphp: Slot [31] registered [ 0.219776] PCI host bridge to bus 0000:00 [ 0.220021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.221024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.222023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.223026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.224020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.225024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.226179] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.228028] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.230292] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.236015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.239076] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.240016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.241011] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.242015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.243455] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.245219] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.248033] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.250807] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.255014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.267013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.270012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.274226] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.281020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.286015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.300014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.307022] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.316016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.322016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.335020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.341522] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.347014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.352015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.364015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.372575] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.377013] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.380013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.391019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.400145] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.407025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.414015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.438026] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.451537] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.469043] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.511028] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.543028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.554746] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.556731] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.559359] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.561490] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.563554] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.571078] iommu: Default domain type: Passthrough [ 0.573405] SCSI subsystem initialized [ 0.574136] ACPI: bus type USB registered [ 0.576090] usbcore: registered new interface driver usbfs [ 0.578159] usbcore: registered new interface driver hub [ 0.580139] usbcore: registered new device driver usb [ 0.582230] pps_core: LinuxPPS API ver. 1 registered [ 0.583008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.586053] PTP clock support registered [ 0.588065] EDAC MC: Ver: 3.0.0 [ 0.590055] PCI: Using ACPI for IRQ routing [ 0.591547] NetLabel: Initializing [ 0.592006] NetLabel: domain hash size = 128 [ 0.592921] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.593078] NetLabel: unlabeled traffic allowed by default [ 0.595046] vgaarb: loaded [ 0.596257] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.597009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.604046] clocksource: Switched to clocksource kvm-clock [ 0.696409] VFS: Disk quotas dquot_6.6.0 [ 0.698082] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.700616] *** VALIDATE ramfs *** [ 0.701845] *** VALIDATE hugetlbfs *** [ 0.703480] pnp: PnP ACPI init [ 0.705713] pnp: PnP ACPI: found 6 devices [ 0.724436] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.727163] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.729396] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.731164] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.733329] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.736067] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.739228] NET: Registered protocol family 2 [ 0.741125] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.744896] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.747349] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.751155] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.753641] TCP: Hash tables configured (established 65536 bind 65536) [ 0.755814] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.758062] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.760897] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.763378] NET: Registered protocol family 1 [ 0.765447] RPC: Registered named UNIX socket transport module. [ 0.766862] RPC: Registered udp transport module. [ 0.767883] RPC: Registered tcp transport module. [ 0.768875] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.770451] NET: Registered protocol family 44 [ 0.771566] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.773260] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.774652] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.776987] PCI: CLS 0 bytes, default 64 [ 0.778153] Unpacking initramfs... [ 2.114949] debug: unmapping init [mem 0xffff9c9cfcc54000-0xffff9c9cfffbffff] [ 2.119856] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.121947] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.125259] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.667613] Initialise system trusted keyrings [ 2.669489] Key type blacklist registered [ 2.671757] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.681389] zbud: loaded [ 2.684660] *** VALIDATE nfs *** [ 2.686024] *** VALIDATE nfs4 *** [ 2.687762] pstore: using deflate compression [ 2.691686] Platform Keyring initialized [ 2.795412] NET: Registered protocol family 38 [ 2.797090] Key type asymmetric registered [ 2.798798] Asymmetric key parser 'x509' registered [ 2.800635] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.803978] io scheduler mq-deadline registered [ 2.805613] io scheduler kyber registered [ 2.807113] io scheduler bfq registered [ 2.808555] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.810602] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.813321] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.815768] ACPI: Power Button [PWRF] [ 2.820816] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.828120] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.850380] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.856603] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.871976] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.900761] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.928170] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.932218] Non-volatile memory driver v1.3 [ 2.934324] Linux agpgart interface v0.103 [ 2.970366] virtio_blk virtio1: [vda] 149792 512-byte logical blocks (76.7 MB/73.1 MiB) [ 2.973601] vda: detected capacity change from 0 to 76693504 [ 2.989500] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.992432] vdb: detected capacity change from 0 to 1073741824 [ 3.009295] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.012247] vdc: detected capacity change from 0 to 2621440000 [ 3.033545] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.037085] vdd: detected capacity change from 0 to 2621440000 [ 3.054530] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.057515] vde: detected capacity change from 0 to 4294967296 [ 3.079659] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.082630] vdf: detected capacity change from 0 to 4294967296 [ 3.097157] libphy: Fixed MDIO Bus: probed [ 3.106744] usbcore: registered new interface driver usbserial_generic [ 3.108926] usbserial: USB Serial support registered for generic [ 3.110623] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.114392] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.116746] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.119543] mousedev: PS/2 mouse device common for all mice [ 3.122693] rtc_cmos 00:05: RTC can wake from S4 [ 3.124488] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.126667] rtc_cmos 00:05: registered as rtc0 [ 3.129622] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.130062] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.135170] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.137168] intel_pstate: CPU model not supported [ 3.144258] hid: raw HID events driver (C) Jiri Kosina [ 3.147140] usbcore: registered new interface driver usbhid [ 3.149320] usbhid: USB HID core driver [ 3.150971] drop_monitor: Initializing network drop monitor service [ 3.153414] Initializing XFRM netlink socket [ 3.155670] NET: Registered protocol family 10 [ 3.159404] Segment Routing with IPv6 [ 3.160978] NET: Registered protocol family 17 [ 3.163090] mpls_gso: MPLS GSO support [ 3.168433] RAS: Correctable Errors collector initialized. [ 3.170584] AVX version of gcm_enc/dec engaged. [ 3.172430] AES CTR mode by8 optimization enabled [ 3.253553] sched_clock: Marking stable (3253519699, 0)->(4182438541, -928918842) [ 3.256910] registered taskstats version 1 [ 3.259054] Loading compiled-in X.509 certificates [ 3.260789] zswap: loaded using pool lzo/zbud [ 3.287811] Key type big_key registered [ 3.300498] Key type encrypted registered [ 3.302356] ima: No TPM chip found, activating TPM-bypass! [ 3.304494] ima: Allocated hash algorithm: sha1 [ 3.305937] ima: No architecture policies found [ 3.307299] evm: Initialising EVM extended attributes: [ 3.309519] evm: security.selinux [ 3.310981] evm: security.ima [ 3.312122] evm: security.capability [ 3.313047] evm: HMAC attrs: 0x1 [ 3.315098] rtc_cmos 00:05: setting system clock to 2026-09-03 20:47:02 UTC (1788468422) [ 3.320983] debug: unmapping init [mem 0xffffffffbb203000-0xffffffffbb3fffff] [ 3.324728] debug: unmapping init [mem 0xffffffffb9f82000-0xffffffffba258fff] [ 3.337097] Write protecting the kernel read-only data: 28672k [ 3.340455] debug: unmapping init [mem 0xffffffffb8603000-0xffffffffb87fffff] [ 3.343385] debug: unmapping init [mem 0xffffffffb8f14000-0xffffffffb8ffffff] [ 3.407704] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.417672] systemd[1]: Detected virtualization kvm. [ 3.420014] systemd[1]: Detected architecture x86-64. [ 3.422286] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.450304] systemd[1]: No hostname configured. [ 3.452407] systemd[1]: Set hostname to . [ 3.455260] random: systemd: uninitialized urandom read (16 bytes read) [ 3.458483] systemd[1]: Initializing machine ID from random generator. [ 3.513428] random: ln: uninitialized urandom read (6 bytes read) [ 3.608861] random: systemd: uninitialized urandom read (16 bytes read) [ 3.611421] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.616146] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.620717] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. Starting Journal Service... Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.166503] device-mapper: uevent: version 1.0.3 [ 4.169126] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.831880] virtio_net virtio0 ens2: renamed from eth0 [ 4.879811] random: fast init done [ 4.906508] scsi host0: ata_piix [ 4.909037] scsi host1: ata_piix [ 4.918761] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.921330] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.271383] dracut-initqueue[590]: RTNETLINK answers: File exists [ 9.817623] random: crng init done [ 9.818557] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.198840] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.275334] printk: systemd: 24 output lines suppressed due to ratelimiting [ 11.496248] SELinux: Disabled at runtime. [ 11.550142] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.557841] systemd[1]: Detected virtualization kvm. [ 11.559629] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.952641] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.956229] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.960332] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.965576] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.970854] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.979548] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.983963] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Control Socket. [ 12.071546] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Remount Root and Kernel File Systems... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.410473] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.715433] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.744894] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.871583] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.880459] EDAC sbridge: Ver: 1.1.2 [ 14.119314] Key type dns_resolver registered [ 14.424335] NFS: Registering the id_resolver key type [ 14.426935] Key type id_resolver registered [ 14.428765] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started dnf makecache --timer. Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg119-server login: [ 39.299296] libcfs: loading out-of-tree module taints kernel. [ 39.323575] Key type ._llcrypt registered [ 39.325364] Key type .llcrypt registered [ 39.373580] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 48.656312] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 49.509488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 49.517443] alg: No test for adler32 (adler32-zlib) [ 51.183972] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 51.757860] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 53.455192] Key type lgssc registered [ 55.373349] Lustre: Echo OBD driver; http://www.lustre.org/ [ 73.075233] hrtimer: interrupt took 23561526 ns [ 78.627038] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 123.008655] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 137.046552] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 137.064824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 138.421331] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 138.447887] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 138.541916] Lustre: lustre-MDT0000: new disk, initializing [ 138.613604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 138.630056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 142.639896] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 159.470960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 159.633846] Lustre: 6494:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 159.692592] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 159.700016] Lustre: Skipped 1 previous similar message [ 159.804653] Lustre: lustre-MDT0001: new disk, initializing [ 159.926048] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 159.961972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 159.982229] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 165.696652] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 169.950426] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 180.304710] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 180.635910] Lustre: lustre-OST0000: new disk, initializing [ 180.641156] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 180.646964] Lustre: 8433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 180.713427] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 186.942904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 186.950296] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 187.074980] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 187.280865] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 200.975325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 201.084775] Lustre: lustre-OST0001: new disk, initializing [ 201.088669] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 201.093778] Lustre: 9504:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 201.150812] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 207.184110] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 210.533479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 210.544088] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 210.613854] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 217.832541] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 230.520790] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 236.365838] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing check_logdir /tmp/testlogs/ [ 241.202445] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing yml_node [ 246.034574] Lustre: DEBUG MARKER: Client: 2.17.58.4 [ 248.643757] Lustre: DEBUG MARKER: MDS: 2.17.58.4 [ 251.186536] Lustre: DEBUG MARKER: OSS: 2.17.58.4 [ 252.890575] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 16:51:10 EDT 2026 [ 271.003689] Lustre: DEBUG MARKER: excepting tests: 21e 32b 32c 32newtarball [ 272.476266] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 287.720122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 287.726788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 287.736217] Lustre: Skipped 3 previous similar messages [ 287.739315] Lustre: Skipped 1 previous similar message [ 292.836657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 292.842442] Lustre: Skipped 5 previous similar messages [ 293.630034] Lustre: server umount lustre-MDT0000 complete [ 297.117260] LustreError: 7443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788468716 with bad export cookie 14850311773853657856 [ 297.133903] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 297.146942] Lustre: 12311:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 297.955520] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 297.962920] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 297.971705] Lustre: Skipped 1 previous similar message [ 297.992708] Lustre: Skipped 1 previous similar message [ 303.072221] Lustre: 12316:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 303.072408] LustreError: 6503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 303.072576] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 303.122262] LustreError: 6503:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 303.668969] Lustre: server umount lustre-MDT0001 complete [ 307.282188] Lustre: server umount lustre-OST0000 complete [ 308.193704] Lustre: 12311:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 310.187157] Lustre: server umount lustre-OST0001 complete [ 317.841847] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 330.680826] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 369.597602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 369.974430] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 370.057211] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 370.209744] Lustre: lustre-MDT0000: new disk, initializing [ 370.310943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 370.333121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 374.985618] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 385.432739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 385.507433] Lustre: 15063:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 385.569466] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 385.572719] Lustre: Skipped 1 previous similar message [ 385.638752] Lustre: lustre-MDT0001: new disk, initializing [ 385.756081] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 385.838081] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 385.852608] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 390.100144] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 399.280028] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 405.543450] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 412.593688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 413.030761] Lustre: lustre-OST0000: new disk, initializing [ 413.035648] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 413.049311] Lustre: 17033:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 413.209914] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 414.906387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 414.922495] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 415.034709] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 419.908278] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 428.932912] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 434.615443] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 434.927272] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 439.220444] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 439.505839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 445.935489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 445.941504] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 445.950524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 445.950558] Lustre: Skipped 1 previous similar message [ 445.953674] Lustre: Skipped 1 previous similar message [ 445.973377] LustreError: Skipped 1 previous similar message [ 447.842884] Lustre: server umount lustre-OST0000 complete [ 452.577317] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 452.586240] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 456.160904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 456.162920] Lustre: Skipped 2 previous similar messages [ 458.069722] Lustre: server umount lustre-MDT0000 complete [ 461.283442] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 461.286908] LustreError: 15075:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 461.311918] Lustre: 18640:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 461.335080] LustreError: 15075:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 462.441019] Lustre: server umount lustre-MDT0001 complete [ 466.244983] Lustre: DEBUG MARKER: == conf-sanity test 0: single mount setup ================ 16:54:44 (1788468884) [ 473.509693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 474.070088] LustreError: 19304:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.154149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 479.077366] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 479.211580] LustreError: 19305:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 484.326985] LustreError: 19304:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 488.853717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 495.549861] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 504.266207] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 511.805371] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 521.146741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 528.723405] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 536.885569] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 559.592923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 559.594952] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 559.604757] Lustre: Skipped 1 previous similar message [ 559.609666] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 561.079582] Lustre: server umount lustre-OST0000 complete [ 570.842675] Lustre: server umount lustre-MDT0000 complete [ 575.011223] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 575.020932] LustreError: 19305:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 575.045672] Lustre: 22148:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 575.086611] LustreError: 19305:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 577.114626] Lustre: server umount lustre-MDT0001 complete [ 594.784489] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 598.073738] Key type lgssc unregistered [ 598.441895] LNet: 22883:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 598.449146] LNetError: 22883:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 598.480487] LNet: Removed LNI 192.168.201.119@tcp [ 599.972183] Key type .llcrypt unregistered [ 599.978088] Key type ._llcrypt unregistered [ 629.428181] Key type ._llcrypt registered [ 629.429978] Key type .llcrypt registered [ 629.554817] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 630.481653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 630.495989] alg: No test for adler32 (adler32-zlib) [ 631.652172] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 631.818222] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 633.463240] Key type lgssc registered [ 635.009545] Lustre: Echo OBD driver; http://www.lustre.org/ [ 644.175786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 644.215644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 645.908952] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 651.792317] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 656.353792] LustreError: 24292:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 656.375627] LustreError: 24292:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 657.383220] LustreError: 24293:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 662.408079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.500649] LustreError: 24292:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 663.081546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 668.668552] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 676.014540] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 681.242906] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 689.357530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 690.126434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 692.218395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:35 to 0x280000401:65) [ 697.062242] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 706.303650] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 722.912504] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 722.921714] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 722.940177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 723.948631] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 723.962490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 724.007100] Lustre: Skipped 1 previous similar message [ 727.888295] Lustre: server umount lustre-OST0000 complete [ 734.178149] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 734.185266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 734.207550] Lustre: Skipped 1 previous similar message [ 734.235354] Lustre: Skipped 1 previous similar message [ 737.837444] Lustre: server umount lustre-MDT0000 complete [ 739.316527] LustreError: 24292:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 739.352508] LustreError: 24292:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 741.927244] LustreError: 24273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788469161 with bad export cookie 13300932663238533494 [ 741.933342] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 741.944972] LustreError: 24273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 742.124390] Lustre: server umount lustre-MDT0001 complete [ 754.656465] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 758.442916] Key type lgssc unregistered [ 758.762784] LNet: 28271:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 758.766877] LNetError: 28271:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 758.779046] LNet: Removed LNI 192.168.201.119@tcp [ 759.967954] Key type .llcrypt unregistered [ 759.970245] Key type ._llcrypt unregistered [ 790.487544] Key type ._llcrypt registered [ 790.490371] Key type .llcrypt registered [ 790.737831] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 793.030803] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 793.094547] alg: No test for adler32 (adler32-zlib) [ 794.221866] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 794.530569] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 796.207154] Key type lgssc registered [ 796.937680] Lustre: Echo OBD driver; http://www.lustre.org/ [ 805.485264] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 805.515193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 807.120449] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 811.809228] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 817.666988] LustreError: 29683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 817.726682] LustreError: 29683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 820.416494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.738726] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 826.498874] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 836.644785] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 843.315398] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 859.396895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 859.856890] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 867.813174] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 876.342949] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 877.301451] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:67 to 0x280000401:97) [ 885.737846] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 885.753087] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 885.783533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 887.789662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 887.811056] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 887.816691] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 891.625425] Lustre: server umount lustre-OST0000 complete [ 897.504789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 897.508178] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 897.526506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 897.539675] Lustre: Skipped 1 previous similar message [ 902.578467] Lustre: server umount lustre-MDT0000 complete [ 903.140193] LustreError: 31770:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 903.163575] LustreError: 31770:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 907.112176] LustreError: 29663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788469326 with bad export cookie 5044116286896657066 [ 907.114339] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 907.121076] LustreError: 29663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 907.552450] Lustre: server umount lustre-MDT0001 complete [ 920.052882] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 923.897411] Key type lgssc unregistered [ 924.364590] LNet: 33578:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 924.382324] LNetError: 33578:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 924.403821] LNet: Removed LNI 192.168.201.119@tcp [ 925.439381] Key type .llcrypt unregistered [ 925.444074] Key type ._llcrypt unregistered [ 952.183025] Key type ._llcrypt registered [ 952.188767] Key type .llcrypt registered [ 952.319421] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 953.461209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 953.545409] alg: No test for adler32 (adler32-zlib) [ 954.691489] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 954.924081] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 956.607237] Key type lgssc registered [ 957.729288] Lustre: Echo OBD driver; http://www.lustre.org/ [ 966.000963] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 966.019678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 967.478256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 971.953133] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 976.867892] LustreError: 34991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 976.893842] LustreError: 34991:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 981.558328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 982.088464] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 987.327656] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 994.447110] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 999.805499] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1006.935581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1007.621322] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1012.744745] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:99 to 0x280000401:129) [ 1016.023784] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1024.347977] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1038.307048] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1038.312548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1038.319131] Lustre: Skipped 1 previous similar message [ 1038.329381] Lustre: Skipped 1 previous similar message [ 1040.416662] Lustre: server umount lustre-OST0000 complete [ 1048.545534] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1048.555841] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1048.557583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1048.578496] Lustre: Skipped 1 previous similar message [ 1050.582620] Lustre: server umount lustre-MDT0000 complete [ 1053.671681] LustreError: 34992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1053.690942] LustreError: 34992:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1054.224378] LustreError: 34972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788469473 with bad export cookie 17136802318050566006 [ 1054.231179] LustreError: 34972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1054.232134] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1054.490510] Lustre: server umount lustre-MDT0001 complete [ 1068.514773] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1072.428076] Key type lgssc unregistered [ 1072.878930] LNet: 38527:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1072.893091] LNetError: 38527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1072.926551] LNet: Removed LNI 192.168.201.119@tcp [ 1074.269527] Key type .llcrypt unregistered [ 1074.276714] Key type ._llcrypt unregistered [ 1100.407409] Key type ._llcrypt registered [ 1100.413756] Key type .llcrypt registered [ 1100.626839] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1102.300975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1102.446456] alg: No test for adler32 (adler32-zlib) [ 1103.465194] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 1103.683494] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1105.359454] Key type lgssc registered [ 1106.731438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1116.940252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1116.958143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1118.383386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1124.624131] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1127.907042] LustreError: 39938:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1127.939422] LustreError: 39938:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1133.035020] LustreError: 39939:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1135.100412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1135.506825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1140.263358] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1147.591504] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1153.140165] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1160.600858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1161.134107] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1163.201235] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:131 to 0x280000401:161) [ 1168.573331] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1175.982508] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1183.719369] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1183.719889] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1183.747321] Lustre: Skipped 1 previous similar message [ 1183.749971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1183.752022] Lustre: Skipped 1 previous similar message [ 1188.813704] Lustre: lustre-OST0000: Not available for connect from 192.168.201.19@tcp (stopping) [ 1188.834089] Lustre: Skipped 1 previous similar message [ 1189.425710] Lustre: server umount lustre-OST0000 complete [ 1212.386074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1212.391831] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1212.405952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1212.409832] Lustre: Skipped 1 previous similar message [ 1214.432680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1214.434227] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1216.241557] Lustre: server umount lustre-MDT0000 complete [ 1220.176458] LustreError: 39920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788469639 with bad export cookie 13932677899533968377 [ 1220.207141] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1226.846305] Lustre: server umount lustre-MDT0001 complete [ 1246.845638] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1250.654099] Key type lgssc unregistered [ 1250.924540] LNet: 43665:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1250.932136] LNetError: 43665:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1250.943321] LNet: Removed LNI 192.168.201.119@tcp [ 1251.855816] Key type .llcrypt unregistered [ 1251.860313] Key type ._llcrypt unregistered [ 1280.604932] Key type ._llcrypt registered [ 1280.606360] Key type .llcrypt registered [ 1280.784094] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1282.637815] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1282.697445] alg: No test for adler32 (adler32-zlib) [ 1283.928351] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 1284.271307] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1286.111807] Key type lgssc registered [ 1287.593752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1299.665695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1299.675896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1301.110675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1308.422499] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1311.727461] LustreError: 45076:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1311.752431] LustreError: 45076:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1316.841322] LustreError: 45075:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1317.309088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1317.672220] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1322.064543] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1330.254944] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1335.918335] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1342.238090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1342.443086] LustreError: 46896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1342.592657] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1346.694875] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:193) [ 1349.410990] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1356.788166] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1363.936284] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1363.947788] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1363.974680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1365.500654] Lustre: server umount lustre-MDT0000 complete [ 1367.525994] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1367.527537] LustreError: 45075:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1367.545934] Lustre: Skipped 1 previous similar message [ 1367.563548] LustreError: 45075:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1368.861777] LustreError: 45055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788469788 with bad export cookie 7569905299913301624 [ 1368.871324] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1368.871800] LustreError: 45055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1369.555427] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.19@tcp (stopping) [ 1369.558433] Lustre: Skipped 1 previous similar message [ 1372.645189] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1372.650037] LustreError: 47657:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1372.656020] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1372.669532] LustreError: 47657:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1374.667620] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.19@tcp (stopping) [ 1374.675751] Lustre: Skipped 1 previous similar message [ 1378.785280] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1379.821045] Lustre: 47859:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1383.394314] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1383.700101] Lustre: server umount lustre-MDT0001 complete [ 1384.935343] Lustre: 47862:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1390.048462] Lustre: 47862:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1395.170592] Lustre: 47859:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1399.586637] Lustre: server umount lustre-OST0000 complete [ 1418.246975] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1421.474855] Key type lgssc unregistered [ 1421.850930] LNet: 48943:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1421.875317] LNetError: 48943:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1421.904382] LNet: Removed LNI 192.168.201.119@tcp [ 1423.018888] Key type .llcrypt unregistered [ 1423.020174] Key type ._llcrypt unregistered [ 1450.930890] Key type ._llcrypt registered [ 1450.932368] Key type .llcrypt registered [ 1451.169250] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1452.325928] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1452.378207] alg: No test for adler32 (adler32-zlib) [ 1453.564227] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 1454.003213] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1455.712800] Key type lgssc registered [ 1457.674318] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1469.552357] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1469.572165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1485.087372] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1490.207322] LustreError: 50328:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 1516.900038] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1523.872709] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1530.944432] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1697.994047] Lustre: server umount lustre-OST0000 complete [ 1716.453921] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1720.415098] Key type lgssc unregistered [ 1720.710757] LNet: 52002:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1720.717665] LNetError: 52002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1720.730288] LNet: Removed LNI 192.168.201.119@tcp [ 1721.797588] Key type .llcrypt unregistered [ 1721.804219] Key type ._llcrypt unregistered [ 1756.045785] Key type ._llcrypt registered [ 1756.049448] Key type .llcrypt registered [ 1756.182741] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1757.380893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1757.577631] alg: No test for adler32 (adler32-zlib) [ 1758.604282] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 1758.821469] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1760.487641] Key type lgssc registered [ 1761.868493] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1771.370768] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1771.383424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1772.627235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1777.646577] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1782.760667] LustreError: 53415:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1782.829734] LustreError: 53415:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1787.886111] LustreError: 53416:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1788.705155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1789.316713] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1794.004202] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1804.401742] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1811.021709] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1819.649357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1820.191429] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1828.311882] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1837.215376] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1839.589747] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1839.600806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1839.602942] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1841.390977] Lustre: server umount lustre-OST0000 complete [ 1845.729072] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1845.736746] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1845.746595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1847.777913] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1847.778561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1851.345380] Lustre: server umount lustre-MDT0000 complete [ 1852.896211] LustreError: 53415:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1852.925323] LustreError: 53415:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1856.100759] LustreError: 55995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788470275 with bad export cookie 17069675833448816877 [ 1856.111984] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1856.120323] LustreError: 55995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1857.703897] Lustre: server umount lustre-MDT0001 complete [ 1872.713654] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1876.571410] Key type lgssc unregistered [ 1876.911621] LNet: 56950:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1876.938212] LNetError: 56950:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1877.990972] LNet: Removed LNI 192.168.201.119@tcp [ 1879.404453] Key type .llcrypt unregistered [ 1879.408157] Key type ._llcrypt unregistered [ 1907.721109] Key type ._llcrypt registered [ 1907.723514] Key type .llcrypt registered [ 1907.819393] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1908.468766] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1908.543188] alg: No test for adler32 (adler32-zlib) [ 1909.659208] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 1909.828995] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1911.464142] Key type lgssc registered [ 1912.935482] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1923.703497] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1923.736929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1924.923390] Lustre: 58362:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1940.383605] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1945.503372] LustreError: 58350:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 1950.177057] Lustre: 58365:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1960.947237] Lustre: 58362:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1966.048487] Lustre: 58365:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1971.171172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1971.175668] Lustre: 58362:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1978.455923] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1981.413264] Lustre: 58362:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 1981.422983] Lustre: 58362:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 1 previous similar message [ 1986.924951] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1994.872809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1995.478324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1995.574269] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2000.885168] LustreError: 58380:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2000.917365] LustreError: 58380:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2001.342367] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2005.991412] LustreError: 58381:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2006.018599] LustreError: 58381:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2011.121070] LustreError: 58380:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2011.145610] LustreError: 58380:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2011.239674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2011.788687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2016.664437] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2025.797977] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2031.824576] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2036.704766] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2036.711279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2036.727390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2037.219295] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2037.230475] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2037.249351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2040.221936] Lustre: server umount lustre-OST0000 complete [ 2067.936657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2067.968324] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2068.006468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2068.023925] Lustre: Skipped 1 previous similar message [ 2070.194889] Lustre: server umount lustre-MDT0000 complete [ 2073.577851] LustreError: 59397:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2073.641355] LustreError: 59397:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2076.778389] LustreError: 58356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788470495 with bad export cookie 17376455558818030092 [ 2076.790727] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2076.804037] LustreError: 58356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2076.881771] Lustre: 61530:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 2076.910879] Lustre: 61530:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 2 previous similar messages [ 2083.861815] Lustre: server umount lustre-MDT0001 complete [ 2096.894735] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2101.356554] Key type lgssc unregistered [ 2101.949648] LNet: 62105:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2101.961880] LNetError: 62105:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2102.002481] LNet: Removed LNI 192.168.201.119@tcp [ 2103.416311] Key type .llcrypt unregistered [ 2103.418623] Key type ._llcrypt unregistered [ 2129.427726] Key type ._llcrypt registered [ 2129.430117] Key type .llcrypt registered [ 2129.533880] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2130.609603] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2130.662084] alg: No test for adler32 (adler32-zlib) [ 2131.936945] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 2132.253517] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 2133.983258] Key type lgssc registered [ 2135.226539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2144.120777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2144.138919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2145.847625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2151.111904] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2154.977744] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2154.988301] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2160.097122] LustreError: 63513:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2160.823644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2161.257499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2166.097878] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2174.870100] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2181.420492] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2189.527468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2189.896874] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2190.927369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:257) [ 2197.156810] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2206.588986] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2226.668509] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2226.681647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2226.688492] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2226.700250] Lustre: Skipped 1 previous similar message [ 2226.733286] Lustre: Skipped 1 previous similar message [ 2230.700796] Lustre: server umount lustre-OST0000 complete [ 2236.898054] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2236.905827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2236.907526] Lustre: Skipped 1 previous similar message [ 2236.924125] Lustre: Skipped 1 previous similar message [ 2241.899527] Lustre: server umount lustre-MDT0000 complete [ 2242.017933] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2242.038246] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2246.124215] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2246.139951] LustreError: 63512:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2246.936243] LustreError: 63493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788470666 with bad export cookie 16613262108670781001 [ 2246.957541] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2247.642470] Lustre: server umount lustre-MDT0001 complete [ 2261.028841] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2265.151318] Key type lgssc unregistered [ 2265.663193] LNet: 67070:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2265.680787] LNetError: 67070:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2265.703346] LNet: Removed LNI 192.168.201.119@tcp [ 2267.248595] Key type .llcrypt unregistered [ 2267.252613] Key type ._llcrypt unregistered [ 2302.194363] Key type ._llcrypt registered [ 2302.197442] Key type .llcrypt registered [ 2302.364551] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2304.344547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2304.542100] alg: No test for adler32 (adler32-zlib) [ 2305.827514] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 2306.199307] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 2307.951200] Key type lgssc registered [ 2309.279732] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2320.952301] Lustre: DEBUG MARKER: == conf-sanity test 5h: start mdt failure at mdt_fs_setup() ========================================================== 17:25:38 (1788470738) [ 2330.052895] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2330.074043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2331.915685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2338.372750] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2342.374837] LustreError: 68772:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2342.405634] LustreError: 68772:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2347.488869] LustreError: 68773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2348.650868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2349.264030] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2355.294756] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2364.262748] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2371.553496] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2379.944816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2380.613932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2389.991482] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2400.437560] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2402.170603] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:289) [ 2408.214577] Lustre: Failing over lustre-MDT0000 [ 2408.442863] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.19@tcp (stopping) [ 2408.697214] Lustre: server umount lustre-MDT0000 complete [ 2410.465972] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2410.481806] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2410.520227] LustreError: 69501:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2412.523730] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2412.529939] LustreError: 68773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2412.553515] LustreError: 68773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2417.633900] LustreError: 71216:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2417.671707] LustreError: 71216:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2420.295080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2420.496680] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2420.748462] Lustre: *** cfs_fail_loc=135, val=0*** [ 2420.890686] LustreError: 71868:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-2) [ 2420.903779] LustreError: 71868:0:(obd_config.c:2100:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -2 [ 2420.921186] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 2420.921186] [ 2420.934507] LustreError: MGC192.168.201.119@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 2420.948509] LustreError: 71856:0:(tgt_mount.c:1948:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 2420.957501] LustreError: 71856:0:(tgt_mount.c:2734:server_fill_super()) Unable to start targets: -2 [ 2420.966441] LustreError: 71856:0:(tgt_mount.c:2192:server_put_super()) no obd lustre-MDT0000 [ 2420.970784] LustreError: 71856:0:(tgt_mount.c:130:server_deregister_mount()) lustre-MDT0000 not registered [ 2421.009337] Lustre: server umount lustre-MDT0000 complete [ 2421.011106] LustreError: 71856:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 2427.881822] LustreError: 68773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2427.899941] LustreError: 68773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 2429.043899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2429.215186] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2429.573172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2429.652952] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2433.999783] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 2435.070846] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2435.131203] Lustre: lustre-MDT0000: Recovery over after 0:02, of 2 clients 2 recovered and 0 were evicted. [ 2435.168038] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:321) [ 2435.804217] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2445.281946] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2445.289822] Lustre: Skipped 1 previous similar message [ 2445.295095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2450.404503] LustreError: 70597:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2450.411723] LustreError: 70597:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2450.674467] Lustre: server umount lustre-OST0000 complete [ 2460.640424] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2460.650356] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2460.651499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2460.662294] Lustre: Skipped 2 previous similar messages [ 2460.673084] Lustre: Skipped 2 previous similar messages [ 2463.251922] Lustre: server umount lustre-MDT0000 complete [ 2467.599321] LustreError: 68752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788470886 with bad export cookie 346191438942916044 [ 2467.605353] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2467.608801] LustreError: 68752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2467.957515] Lustre: server umount lustre-MDT0001 complete [ 2483.783227] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2488.042994] Key type lgssc unregistered [ 2488.437911] LNet: 73647:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2488.446738] LNetError: 73647:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2489.512618] LNet: Removed LNI 192.168.201.119@tcp [ 2491.083729] Key type .llcrypt unregistered [ 2491.086275] Key type ._llcrypt unregistered [ 2519.862395] Key type ._llcrypt registered [ 2519.864667] Key type .llcrypt registered [ 2519.990501] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2521.157157] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2521.266451] alg: No test for adler32 (adler32-zlib) [ 2522.502532] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 2522.768070] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 2524.516110] Key type lgssc registered [ 2526.128072] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2538.273603] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2538.300970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2539.832525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2545.680328] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2549.220847] LustreError: 75061:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2549.252110] LustreError: 75061:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2554.351838] LustreError: 75062:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2557.229777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2557.886231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2564.218957] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2573.548200] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2580.633304] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2587.818382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2588.273943] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2595.630609] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2603.763878] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2609.808119] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:353) [ 2620.207238] Lustre: Failing over lustre-MDT0000 [ 2620.394215] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2620.413550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2620.415560] Lustre: Skipped 2 previous similar messages [ 2620.438858] Lustre: Skipped 2 previous similar messages [ 2620.712284] Lustre: server umount lustre-MDT0000 complete [ 2624.994982] LustreError: 77504:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2630.097783] LustreError: 75056:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2630.123333] LustreError: 75056:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2630.635974] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2630.659163] Lustre: 77950:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 2633.549335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2634.058966] Lustre: *** cfs_fail_loc=a05, val=0*** [ 2634.253992] LustreError: 78176:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-9) [ 2634.269081] LustreError: 78176:0:(obd_config.c:2100:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -9 [ 2634.276506] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 2634.276506] [ 2634.292018] LustreError: MGC192.168.201.119@tcp: Confguration from log lustre-MDT0000 failed from MGS -9. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 2634.309823] LustreError: 78165:0:(tgt_mount.c:1948:server_start_targets()) failed to start server lustre-MDT0000: -9 [ 2634.318347] LustreError: 78165:0:(tgt_mount.c:2734:server_fill_super()) Unable to start targets: -9 [ 2634.325399] LustreError: 78165:0:(tgt_mount.c:2192:server_put_super()) no obd lustre-MDT0000 [ 2634.359956] LustreError: 78165:0:(tgt_mount.c:130:server_deregister_mount()) lustre-MDT0000 not registered [ 2634.402217] Lustre: server umount lustre-MDT0000 complete [ 2634.404134] LustreError: 78165:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -9 [ 2635.207698] LustreError: 75057:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2635.237632] LustreError: 75057:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2642.994388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2643.155989] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2643.527319] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2643.613869] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2645.451774] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 2648.588126] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2648.723543] Lustre: lustre-MDT0000: Recovery over after 0:03, of 2 clients 2 recovered and 0 were evicted. [ 2648.798861] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:385) [ 2649.048671] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2658.788812] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2658.799488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2658.807334] Lustre: Skipped 1 previous similar message [ 2663.873545] Lustre: server umount lustre-OST0000 complete [ 2669.027804] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2669.049147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2669.051670] Lustre: Skipped 1 previous similar message [ 2674.144142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2674.156848] Lustre: Skipped 1 previous similar message [ 2679.264918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2679.275701] Lustre: Skipped 1 previous similar message [ 2682.337754] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2682.572847] Lustre: server umount lustre-MDT0000 complete [ 2684.389652] LustreError: 75058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2684.420097] LustreError: 75058:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 2686.542141] LustreError: 78555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471105 with bad export cookie 13516608937521473724 [ 2686.551050] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2686.559208] LustreError: 78555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2687.031630] Lustre: server umount lustre-MDT0001 complete [ 2699.603889] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2702.722261] Key type lgssc unregistered [ 2703.155058] LNet: 79959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2703.165414] LNetError: 79959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2703.197861] LNet: Removed LNI 192.168.201.119@tcp [ 2703.871658] Key type .llcrypt unregistered [ 2703.873759] Key type ._llcrypt unregistered [ 2717.627855] Key type ._llcrypt registered [ 2717.631257] Key type .llcrypt registered [ 2717.727294] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2718.058764] Key type .llcrypt unregistered [ 2718.060511] Key type ._llcrypt unregistered [ 2742.872734] Key type ._llcrypt registered [ 2742.880204] Key type .llcrypt registered [ 2743.059370] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2744.914769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2744.952743] alg: No test for adler32 (adler32-zlib) [ 2746.077950] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 2746.480702] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 2748.170093] Key type lgssc registered [ 2749.912374] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2760.014690] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2760.047080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2761.627201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2766.834431] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2770.923517] LustreError: 81851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2770.951384] LustreError: 81851:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2776.037053] LustreError: 81850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2777.170682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2777.727958] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2783.548596] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2792.243839] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2798.362210] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2805.587399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2805.950436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2806.985490] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:417) [ 2813.641151] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2821.774468] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2832.866717] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2832.868711] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2832.901073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2837.985340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2837.999178] Lustre: Skipped 1 previous similar message [ 2838.726427] Lustre: server umount lustre-OST0000 complete [ 2843.105182] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2843.121382] Lustre: Skipped 1 previous similar message [ 2843.129980] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2843.134718] Lustre: Skipped 1 previous similar message [ 2848.224386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2848.230115] Lustre: Skipped 1 previous similar message [ 2849.287072] Lustre: server umount lustre-MDT0000 complete [ 2853.211989] LustreError: 81833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471272 with bad export cookie 9856419012374899517 [ 2853.214574] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2853.222026] LustreError: 81833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2853.550358] Lustre: server umount lustre-MDT0001 complete [ 2863.351624] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 2867.189736] Key type lgssc unregistered [ 2867.568284] LNet: 85384:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2867.588434] LNetError: 85384:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2867.620110] LNet: Removed LNI 192.168.201.119@tcp [ 2868.959678] Key type .llcrypt unregistered [ 2868.962820] Key type ._llcrypt unregistered [ 2897.685533] Key type ._llcrypt registered [ 2897.688434] Key type .llcrypt registered [ 2897.783611] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2898.742566] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2898.896149] alg: No test for adler32 (adler32-zlib) [ 2900.053695] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 2900.340746] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 2902.059241] Key type lgssc registered [ 2903.805816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2912.386417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2912.419936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2913.885522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2918.666930] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2924.008022] LustreError: 86792:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2924.034785] LustreError: 86792:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2927.715744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2928.123417] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2932.431517] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2940.111472] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2945.619372] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2952.901990] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2953.280255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2959.420090] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:419 to 0x280000401:449) [ 2959.575311] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2967.155365] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2974.694826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2974.711214] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2974.715066] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2974.725880] Lustre: Skipped 1 previous similar message [ 2974.751488] LustreError: Skipped 1 previous similar message [ 2976.989054] Lustre: server umount lustre-OST0000 complete [ 2984.416334] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2984.433289] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2984.448244] Lustre: Skipped 1 previous similar message [ 2984.455688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2987.562425] Lustre: server umount lustre-MDT0000 complete [ 2990.049415] LustreError: 86792:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2990.071646] LustreError: 86792:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2991.641844] LustreError: 86773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471410 with bad export cookie 7999896571408431910 [ 2991.644423] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2991.648030] LustreError: 86773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2991.874660] Lustre: server umount lustre-MDT0001 complete [ 3001.359827] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 3003.255639] Key type lgssc unregistered [ 3003.509416] LNet: 90329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3003.515333] LNetError: 90329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3003.531268] LNet: Removed LNI 192.168.201.119@tcp [ 3004.202940] Key type .llcrypt unregistered [ 3004.204231] Key type ._llcrypt unregistered [ 3029.695674] Key type ._llcrypt registered [ 3029.697679] Key type .llcrypt registered [ 3029.853195] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3030.754300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3030.973949] alg: No test for adler32 (adler32-zlib) [ 3032.180425] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 3032.458971] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 3034.159304] Key type lgssc registered [ 3035.526986] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3044.399846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3044.418538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3045.728812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3051.327841] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3055.100804] LustreError: 91739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3055.116386] LustreError: 91739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3060.193259] LustreError: 91740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3062.055780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3062.765685] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3067.769963] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3077.462767] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3084.578805] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3092.381105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3092.875385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3100.518039] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3109.587824] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3115.455910] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:419 to 0x280000401:481) [ 3130.853576] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3130.855917] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3130.898335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3130.917154] Lustre: Skipped 1 previous similar message [ 3135.156262] Lustre: server umount lustre-OST0000 complete [ 3139.552137] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3139.559718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3139.572567] Lustre: Skipped 1 previous similar message [ 3139.587031] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3141.089569] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3141.091033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3141.127643] Lustre: Skipped 1 previous similar message [ 3145.902558] Lustre: server umount lustre-MDT0000 complete [ 3146.222810] LustreError: 91740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3146.239419] LustreError: 91740:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3150.169593] LustreError: 93580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471569 with bad export cookie 11886750006379672341 [ 3150.174733] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3150.185500] LustreError: 93580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3150.601502] Lustre: server umount lustre-MDT0001 complete [ 3162.183155] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 3165.181717] Key type lgssc unregistered [ 3165.527795] LNet: 95296:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3165.534046] LNetError: 95296:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3165.546742] LNet: Removed LNI 192.168.201.119@tcp [ 3166.574165] Key type .llcrypt unregistered [ 3166.577080] Key type ._llcrypt unregistered [ 3190.692219] Key type ._llcrypt registered [ 3190.694876] Key type .llcrypt registered [ 3190.812590] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3192.174198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3192.202915] alg: No test for adler32 (adler32-zlib) [ 3193.460239] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 3193.766566] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 3195.551279] Key type lgssc registered [ 3197.213479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3205.566820] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3205.580144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3206.697871] Lustre: 96689:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3222.178877] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3227.295363] LustreError: 96677:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 3231.717855] Lustre: 96692:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3242.666789] Lustre: 96689:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3248.095682] Lustre: 96692:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3252.196360] Lustre: 96689:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3252.959492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3260.686704] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3262.435321] Lustre: 96689:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3262.452593] Lustre: 96689:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 1 previous similar message [ 3269.234648] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3274.974371] Lustre: server umount lustre-OST0000 complete [ 3284.350482] Lustre: DEBUG MARKER: == conf-sanity test 10a: find lctl param broken symlinks ========================================================== 17:41:41 (1788471701) [ 3291.417016] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3291.889588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3296.279247] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3297.255627] LustreError: 98325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3297.282991] LustreError: 98325:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3302.369560] LustreError: 98324:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3306.184892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3306.744533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3313.015993] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3322.611310] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3328.638269] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3336.046218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3336.598337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3337.615255] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:484 to 0x280000401:513) [ 3343.811820] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3351.773684] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3366.378772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3366.380292] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3366.408936] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3369.899176] Lustre: server umount lustre-OST0000 complete [ 3376.608168] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3376.611285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3376.626224] Lustre: Skipped 2 previous similar messages [ 3376.646600] Lustre: Skipped 2 previous similar messages [ 3381.730094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3381.737013] Lustre: Skipped 1 previous similar message [ 3385.830100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3388.383240] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3388.503512] Lustre: server umount lustre-MDT0000 complete [ 3390.949110] LustreError: 98319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3390.999386] LustreError: 98319:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3392.087679] LustreError: 101263:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471811 with bad export cookie 3912112481326520150 [ 3392.100177] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3392.100648] LustreError: 101263:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3392.331927] Lustre: server umount lustre-MDT0001 complete [ 3403.627645] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 3406.685576] Key type lgssc unregistered [ 3407.063037] LNet: 102015:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3407.068353] LNetError: 102015:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3408.107752] LNet: Removed LNI 192.168.201.119@tcp [ 3408.922133] Key type .llcrypt unregistered [ 3408.923787] Key type ._llcrypt unregistered [ 3453.267926] Key type ._llcrypt registered [ 3453.270966] Key type .llcrypt registered [ 3453.468961] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3455.290731] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3455.504327] alg: No test for adler32 (adler32-zlib) [ 3456.969373] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 3457.343199] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 3459.167647] Key type lgssc registered [ 3460.887679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3471.221394] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3471.259469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3472.877606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3476.465876] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3483.105436] LustreError: 104202:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3483.151555] LustreError: 104202:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3485.073174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3485.556395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3491.011484] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3499.828392] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3506.390179] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3513.358798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3513.865089] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3518.020289] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:484 to 0x280000401:545) [ 3520.743519] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3527.848335] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3538.403457] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3538.413775] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.433691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3540.966204] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3540.969313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3540.990648] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3541.013022] Lustre: Skipped 1 previous similar message [ 3543.934697] Lustre: server umount lustre-OST0000 complete [ 3551.204371] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3551.208379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3551.218777] Lustre: Skipped 1 previous similar message [ 3554.760754] Lustre: server umount lustre-MDT0000 complete [ 3556.321631] LustreError: 104196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3556.334695] LustreError: 104196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3558.130660] LustreError: 104183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788471977 with bad export cookie 12853217713936615066 [ 3558.144624] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3558.407949] Lustre: server umount lustre-MDT0001 complete [ 3568.896417] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 3572.260752] Key type lgssc unregistered [ 3572.516395] LNet: 107742:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3572.523169] LNetError: 107742:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3572.543510] LNet: Removed LNI 192.168.201.119@tcp [ 3573.621765] Key type .llcrypt unregistered [ 3573.624334] Key type ._llcrypt unregistered [ 3594.095612] Key type ._llcrypt registered [ 3594.101153] Key type .llcrypt registered [ 3594.238790] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3595.453287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3595.555532] alg: No test for adler32 (adler32-zlib) [ 3596.723859] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 3596.953914] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 3598.631206] Key type lgssc registered [ 3599.939837] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3607.759186] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3607.782381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3608.891195] Lustre: 108993:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3624.351590] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3629.471520] LustreError: 108981:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 3634.144822] Lustre: 108996:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3644.921231] Lustre: 108993:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3650.016163] Lustre: 108996:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3655.136103] Lustre: 108993:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3655.199712] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3662.427976] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3665.376282] Lustre: 108993:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3665.388240] Lustre: 108993:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 1 previous similar message [ 3670.732544] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3678.337586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3678.790908] LustreError: MGC192.168.201.119@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 3678.810201] LustreError: 110011:0:(tgt_mount.c:1948:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 3678.838198] LustreError: 110011:0:(tgt_mount.c:2734:server_fill_super()) Unable to start targets: -2 [ 3678.849195] LustreError: 110011:0:(tgt_mount.c:2192:server_put_super()) no obd lustre-MDT0000 [ 3678.989932] Lustre: server umount lustre-MDT0000 complete [ 3678.995827] LustreError: 110011:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 3696.095867] Lustre: 108317:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788472099/real 1788472099] req@000000001482a7b2 x1875348832653568/t0(0) o400->MGC192.168.201.119@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788472115 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3696.120939] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3706.337102] Lustre: 110445:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3706.354196] Lustre: 110445:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 2 previous similar messages [ 3716.694498] Lustre: server umount lustre-OST0000 complete [ 3727.193610] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 3737.125705] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3777.037329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3777.237250] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3777.258261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3777.314471] Lustre: lustre-MDT0000: new disk, initializing [ 3777.385471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3777.395536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3783.324917] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3797.351434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3797.502067] Lustre: 112812:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3797.640795] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3797.649943] Lustre: Skipped 1 previous similar message [ 3797.781667] Lustre: lustre-MDT0001: new disk, initializing [ 3797.935597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3797.998345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3798.018840] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3803.641614] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3813.574797] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3820.459646] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3828.360653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3828.661512] Lustre: lustre-OST0000: new disk, initializing [ 3828.665531] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3828.669748] Lustre: 114783:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3828.779820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3836.050950] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3838.605880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3838.619205] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3838.732237] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3846.125911] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3851.469614] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3851.672713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3856.685223] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3856.889659] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3859.425260] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3859.428783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3859.448323] Lustre: Skipped 1 previous similar message [ 3864.551388] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3864.569427] Lustre: Skipped 2 previous similar messages [ 3865.461229] Lustre: server umount lustre-OST0000 complete [ 3869.664447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3869.670971] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3869.677968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3869.684983] Lustre: Skipped 1 previous similar message [ 3874.785052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3874.791650] Lustre: Skipped 2 previous similar messages [ 3875.908626] Lustre: server umount lustre-MDT0000 complete [ 3879.860701] LustreError: 112803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788472299 with bad export cookie 17675996714243914785 [ 3879.872801] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3879.876074] LustreError: 112803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3880.518834] Lustre: server umount lustre-MDT0001 complete [ 3889.552184] Lustre: DEBUG MARKER: == conf-sanity test 18: check mkfs creates large journals ========================================================== 17:51:47 (1788472307) [ 3892.561369] Lustre: DEBUG MARKER: use device /dev/mapper/mds1_flakey with MIN=2000000 [ 3911.811214] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 3923.011722] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3958.537539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3958.846594] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3958.878235] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3958.990779] Lustre: lustre-MDT0000: new disk, initializing [ 3959.117432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3959.136268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3964.624705] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3976.238056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3976.444875] Lustre: 119679:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3976.597718] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3976.600203] Lustre: Skipped 1 previous similar message [ 3976.683540] Lustre: lustre-MDT0001: new disk, initializing [ 3976.770625] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3976.807806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3976.813579] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3980.587284] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3989.726735] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3998.135478] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4004.957162] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4005.245993] Lustre: lustre-OST0000: new disk, initializing [ 4005.254184] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4005.261506] Lustre: 121649:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4005.400880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4012.780994] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4013.799836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4013.834327] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4013.948540] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4023.405464] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4026.139427] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4026.308550] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4030.833302] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4031.166295] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4034.532524] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4034.538860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4034.550734] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4034.564510] Lustre: Skipped 1 previous similar message [ 4034.598696] Lustre: Skipped 1 previous similar message [ 4039.721473] Lustre: server umount lustre-OST0000 complete [ 4044.772883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4044.796405] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4044.809374] Lustre: Skipped 1 previous similar message [ 4050.152130] Lustre: server umount lustre-MDT0000 complete [ 4054.588631] LustreError: 119674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788472473 with bad export cookie 17675996714243915681 [ 4054.590427] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4054.596902] LustreError: 119674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4054.936872] Lustre: server umount lustre-MDT0001 complete [ 4064.037437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4064.556480] LustreError: 123868:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4064.577444] LustreError: 123868:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4064.659801] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4069.857469] LustreError: 123869:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4070.213705] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4073.953935] LustreError: 123868:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4079.073847] LustreError: 123869:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4079.555797] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4085.376969] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4093.804242] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4100.379528] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4107.943796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4115.774974] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4124.127504] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4132.105320] Lustre: DEBUG MARKER: Success: mkfs creates large journals. Size: 78M [ 4140.003035] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4140.011949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4140.032881] Lustre: Skipped 1 previous similar message [ 4140.039454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4140.042771] Lustre: Skipped 2 previous similar messages [ 4151.263213] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4151.528134] Lustre: server umount lustre-OST0000 complete [ 4156.387590] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4156.419891] Lustre: Skipped 2 previous similar messages [ 4161.940545] Lustre: server umount lustre-MDT0000 complete [ 4166.113165] LustreError: 123848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788472585 with bad export cookie 17675996714243916577 [ 4166.123801] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4166.125856] LustreError: 123848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4166.744985] Lustre: server umount lustre-MDT0001 complete [ 4178.595422] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4181.784078] Key type lgssc unregistered [ 4182.098873] LNet: 127502:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4182.112623] LNetError: 127502:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4182.130531] LNet: Removed LNI 192.168.201.119@tcp [ 4183.136541] Key type .llcrypt unregistered [ 4183.138506] Key type ._llcrypt unregistered [ 4203.787245] Key type ._llcrypt registered [ 4203.789276] Key type .llcrypt registered [ 4203.868319] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 4221.104943] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 4222.224752] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4222.292076] alg: No test for adler32 (adler32-zlib) [ 4223.598166] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 4223.853372] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 4225.567399] Key type lgssc registered [ 4226.966554] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4266.645890] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4266.692859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4268.041480] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4268.067480] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4268.194922] Lustre: lustre-MDT0000: new disk, initializing [ 4268.302095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4268.353215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4274.039529] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4287.708118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4287.822905] Lustre: 130653:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4287.970829] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4287.979801] Lustre: Skipped 1 previous similar message [ 4288.106209] Lustre: lustre-MDT0001: new disk, initializing [ 4288.391608] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4288.520548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4288.536606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4293.798982] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4304.905882] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4311.699498] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4318.919359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4319.193189] Lustre: lustre-OST0000: new disk, initializing [ 4319.207746] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4319.225652] Lustre: 132619:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4319.345466] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4326.177158] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4329.596484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4329.601311] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4329.720420] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4335.929094] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4340.740353] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4340.958455] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4346.585350] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4346.802751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4350.433940] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4350.438328] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4350.442676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4350.455317] Lustre: Skipped 1 previous similar message [ 4355.122725] Lustre: server umount lustre-OST0000 complete [ 4360.163034] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4360.177046] LustreError: Skipped 1 previous similar message [ 4360.180832] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4360.197422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4360.211331] Lustre: Skipped 1 previous similar message [ 4367.300808] Lustre: server umount lustre-MDT0000 complete [ 4370.905115] LustreError: 130647:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788472790 with bad export cookie 4644214167256177300 [ 4370.907410] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4370.913269] LustreError: 130647:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4371.293046] Lustre: server umount lustre-MDT0001 complete [ 4379.906547] Lustre: DEBUG MARKER: == conf-sanity test 19a: start/stop MDS without OSTs ===== 17:59:57 (1788472797) [ 4386.832486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4387.617465] LustreError: 135034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4387.638547] LustreError: 135034:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 4387.677090] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4392.936317] LustreError: 135035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4393.143843] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4397.046917] LustreError: 135034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4402.045392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4402.145241] LustreError: 135035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4402.439450] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4407.864356] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4416.088223] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4422.039963] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4427.234275] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4427.237703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4427.241412] Lustre: Skipped 2 previous similar messages [ 4427.249964] Lustre: Skipped 2 previous similar messages [ 4430.696461] Lustre: server umount lustre-MDT0000 complete [ 4433.381822] LustreError: 135034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4433.395905] LustreError: 135034:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4434.166316] LustreError: 135015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788472853 with bad export cookie 4644214167256178196 [ 4434.174470] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4434.439340] Lustre: server umount lustre-MDT0001 complete [ 4441.772583] Lustre: DEBUG MARKER: == conf-sanity test 19b: start/stop OSTs without MDS ===== 18:00:59 (1788472859) [ 4447.744909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4447.843861] Lustre: 137474:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4453.346198] Lustre: 137477:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4458.473084] Lustre: 137474:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4463.332396] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4463.584679] Lustre: 137477:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4468.447565] LustreError: 137462:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 4468.704149] Lustre: 137474:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4478.948382] Lustre: 137474:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4478.960338] Lustre: 137474:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 1 previous similar message [ 4494.239812] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4499.426606] Lustre: 137475:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4499.440862] Lustre: 137475:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 3 previous similar messages [ 4501.426684] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4510.547622] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4512.884535] Lustre: server umount lustre-OST0000 complete [ 4521.986873] Lustre: DEBUG MARKER: == conf-sanity test 20: remount ro,rw mounts work and doesn't break /etc/mtab ========================================================== 18:02:19 (1788472939) [ 4529.640521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4530.062774] LustreError: 138914:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4530.149723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4535.502935] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4544.663446] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4549.665553] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4557.497267] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4563.368876] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4570.347298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4570.597073] LustreError: 140736:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4570.626372] LustreError: 140736:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4570.795491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4570.798270] Lustre: Skipped 1 previous similar message [ 4577.636464] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4586.365903] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4595.682632] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4595.688479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4595.694746] Lustre: Skipped 2 previous similar messages [ 4595.707554] Lustre: Skipped 4 previous similar messages [ 4601.121195] Lustre: server umount lustre-MDT0000 complete [ 4605.448793] LustreError: 138894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473024 with bad export cookie 4644214167256178903 [ 4605.451380] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4605.477245] LustreError: 138894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 10 previous similar messages [ 4605.923996] LustreError: 138911:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4605.925040] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4605.942436] LustreError: 138911:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4606.203705] Lustre: server umount lustre-MDT0001 complete [ 4612.909500] Lustre: server umount lustre-OST0000 complete [ 4621.235309] Lustre: DEBUG MARKER: == conf-sanity test 21a: start mds before ost, stop ost first ========================================================== 18:03:59 (1788473039) [ 4627.391177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4634.065817] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4645.974767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4646.369852] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4646.381935] Lustre: Skipped 1 previous similar message [ 4651.537320] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4660.090305] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4665.822552] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4673.643275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4682.139231] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4692.252327] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4697.855442] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4698.728633] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4699.077286] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4705.112953] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4705.457628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4708.328379] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.333651] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4708.335516] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4708.375431] Lustre: Skipped 1 previous similar message [ 4714.129656] Lustre: server umount lustre-OST0000 complete [ 4718.049945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4718.060924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4718.069165] Lustre: Skipped 3 previous similar messages [ 4732.384033] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4732.536470] Lustre: server umount lustre-MDT0000 complete [ 4733.920626] LustreError: 142567:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4733.960905] LustreError: 142567:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 4736.053710] LustreError: 142554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473155 with bad export cookie 4644214167256180016 [ 4736.070404] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4736.391244] Lustre: server umount lustre-MDT0001 complete [ 4744.519821] Lustre: DEBUG MARKER: == conf-sanity test 21b: start ost before mds, stop mds first ========================================================== 18:06:02 (1788473162) [ 4751.293506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4751.523324] Lustre: 146749:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4751.539135] Lustre: 146749:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 2 previous similar messages [ 4767.136135] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4772.255534] LustreError: 146737:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 4797.983359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4797.997521] Lustre: Skipped 1 previous similar message [ 4805.141656] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4814.873402] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4817.396412] Lustre: 146750:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4817.417841] Lustre: 146750:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 12 previous similar messages [ 4822.503441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4822.969569] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4826.950421] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4836.527871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4841.932577] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4849.952689] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4856.900385] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4862.041189] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4862.388306] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4866.805345] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4867.200444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4872.673133] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4872.681436] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4872.689765] Lustre: Skipped 2 previous similar messages [ 4872.696763] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4872.705901] Lustre: Skipped 5 previous similar messages [ 4875.489370] Lustre: server umount lustre-MDT0000 complete [ 4878.306980] LustreError: 147781:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4878.341351] LustreError: 147781:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 4880.450974] LustreError: 146745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473299 with bad export cookie 4644214167256181094 [ 4880.461791] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4880.470034] LustreError: 146745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 10 previous similar messages [ 4880.907982] Lustre: server umount lustre-MDT0001 complete [ 4897.025870] Lustre: server umount lustre-OST0000 complete [ 4906.899964] Lustre: DEBUG MARKER: == conf-sanity test 21c: start mds between two osts, stop mds last ========================================================== 18:08:44 (1788473324) [ 4914.301198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4930.020962] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4935.199979] LustreError: 150944:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 4950.499750] Lustre: 150958:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 4950.531220] Lustre: 150958:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 10 previous similar messages [ 4967.757799] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4974.085861] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4980.702579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4981.314683] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4986.365976] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4995.756715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5000.746799] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5009.386487] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5015.501800] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5022.720916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5022.992933] Lustre: lustre-OST0001: new disk, initializing [ 5023.004679] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5023.017370] Lustre: 153820:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5030.846391] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5032.601625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5032.616461] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5032.782059] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5041.481975] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5046.812742] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 5047.031754] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5052.073941] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5052.355601] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5057.505805] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5057.515989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5057.534367] Lustre: Skipped 2 previous similar messages [ 5057.542947] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5057.547318] Lustre: Skipped 3 previous similar messages [ 5060.585693] Lustre: server umount lustre-OST0000 complete [ 5070.817884] Lustre: server umount lustre-OST0001 complete [ 5077.984565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5077.991430] LustreError: Skipped 3 previous similar messages [ 5081.211780] Lustre: server umount lustre-MDT0000 complete [ 5084.937869] LustreError: 150950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473504 with bad export cookie 4644214167256182116 [ 5084.939655] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5084.946224] LustreError: 150950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5085.228844] Lustre: server umount lustre-MDT0001 complete [ 5110.269292] Lustre: DEBUG MARKER: == conf-sanity test 21d: start mgs then ost and then mds ========================================================== 18:12:08 (1788473528) [ 5112.028430] Lustre: DEBUG MARKER: SKIP: conf-sanity test_21d need separate mgs device [ 5113.812080] Lustre: DEBUG MARKER: SKIP: conf-sanity test_21e skipping excluded test 21e [ 5115.593803] Lustre: DEBUG MARKER: == conf-sanity test 22: start a client before osts (should return errs) ========================================================== 18:12:13 (1788473533) [ 5123.221972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5123.624508] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 5123.672219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5124.007829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5124.021595] Lustre: Skipped 6 previous similar messages [ 5129.288207] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5138.536443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5138.629455] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5138.675789] Lustre: 157382:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5144.488471] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5153.359828] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5159.474622] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5166.442518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5166.644600] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 5174.117978] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5176.587285] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:161) [ 5182.264289] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5187.468602] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5187.694464] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5192.283314] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5192.523496] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5197.291307] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5197.292126] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5197.295963] LustreError: Skipped 1 previous similar message [ 5197.299031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5197.308804] Lustre: Skipped 6 previous similar messages [ 5197.329790] Lustre: Skipped 8 previous similar messages [ 5200.572396] Lustre: server umount lustre-OST0000 complete [ 5222.057912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5228.748208] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5236.580673] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5237.676237] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.690526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.695526] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5237.739686] Lustre: Skipped 1 previous similar message [ 5243.141903] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5243.478406] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5250.130216] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5250.396323] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5256.953676] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c0fc7684000.ost_server_uuid 50 [ 5259.466392] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c0fc7684000.ost_server_uuid in FULL state after 0 sec [ 5259.614529] LustreError: 157388:0:(ldlm_lockd.c:727:ldlm_handle_ast_error()) ### client (nid 192.168.201.19@tcp) returned error from blocking AST (req@000000008752c30b x1875349490383872 status -107 rc -107), evict it ns: mdt-lustre-MDT0000_UUID lock: 000000004f724db7/0x4073902dd4243704 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.19@tcp remote: 0x6964e635c1c678e2 expref: 5 pid: 157390 timeout: 5359 lvb_type: 0 lru_score: 0 lru_type: 0 [ 5259.650901] LustreError: lustre-MDT0000: A client on nid 192.168.201.19@tcp was evicted due to a lock blocking callback time out: rc -107 [ 5259.658117] LustreError: 157378:0:(ldlm_lockd.c:255:expired_lock_main()) ### lock callback timer expired after 0s: evicting client at 192.168.201.19@tcp ns: mdt-lustre-MDT0000_UUID lock: 000000004f724db7/0x4073902dd4243704 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.19@tcp remote: 0x6964e635c1c678e2 expref: 6 pid: 157390 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 [ 5268.452244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5268.462713] LustreError: Skipped 1 previous similar message [ 5271.494618] Lustre: server umount lustre-OST0000 complete [ 5282.711466] Lustre: server umount lustre-MDT0000 complete [ 5283.809351] LustreError: 157394:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5283.839418] LustreError: 157394:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 5287.037768] LustreError: 157373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473706 with bad export cookie 4644214167256183348 [ 5287.052091] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5287.053266] LustreError: 157373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5293.794506] Lustre: server umount lustre-MDT0001 complete [ 5306.486779] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5309.913340] Key type lgssc unregistered [ 5310.380138] LNet: 163501:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5310.402482] LNetError: 163501:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5310.424104] LNet: Removed LNI 192.168.201.119@tcp [ 5311.393802] Key type .llcrypt unregistered [ 5311.396426] Key type ._llcrypt unregistered [ 5335.222181] Key type ._llcrypt registered [ 5335.232795] Key type .llcrypt registered [ 5335.342710] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5336.425066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5336.447181] alg: No test for adler32 (adler32-zlib) [ 5337.471517] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 5337.690759] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5339.321239] Key type lgssc registered [ 5340.214196] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5347.736162] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5347.743580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5349.155047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5356.012685] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5359.610342] LustreError: 164909:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5359.646177] LustreError: 164909:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5364.706646] LustreError: 164910:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5369.827251] LustreError: 164909:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5371.029129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5371.596173] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5376.825558] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5385.080529] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5390.992128] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5399.174550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5399.608915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5406.387428] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5414.775183] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5419.059427] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:193) [ 5426.032763] Lustre: Failing over lustre-MDT0000 [ 5426.323470] Lustre: server umount lustre-MDT0000 complete [ 5426.668894] LustreError: 164906:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5427.686423] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5427.702620] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5429.219349] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5429.233037] Lustre: Skipped 1 previous similar message [ 5434.337383] LustreError: 164910:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5434.340112] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5434.365792] LustreError: 164910:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 5434.381406] Lustre: 167608:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 5439.459158] Lustre: 167607:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 5444.584523] Lustre: 167608:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 5444.586889] LustreError: 164905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5444.631720] LustreError: 164905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 5446.829295] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5447.406244] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5447.483467] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 5452.153184] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5452.788228] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 5452.796109] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5454.559567] Lustre: lustre-MDT0000: Denying connection for new client aa8292dc-1420-4fc2-a1f9-7231541ebccf (at 192.168.201.19@tcp), waiting for 2 known clients (1 recovered, 0 in progress, and 0 evicted) to recover in 1:07 [ 5463.024052] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5463.032749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5463.052775] Lustre: Skipped 1 previous similar message [ 5463.074189] Lustre: Skipped 1 previous similar message [ 5468.136515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5468.143420] Lustre: Skipped 1 previous similar message [ 5468.498570] Lustre: server umount lustre-OST0000 complete [ 5472.460749] LustreError: 168643:0:(ldlm_lib.c:3043:target_stop_recovery_thread()) lustre-MDT0000: Aborting recovery [ 5472.466929] Lustre: 167992:0:(ldlm_lib.c:2443:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 5472.481560] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 5472.505702] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 5472.520417] LustreError: 167992:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000580c306d x1875350657222528/t0(0) o700->lustre-MDT0001-osp-MDT0000@0@lo:30/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5472.545829] LustreError: 167992:0:(fid_request.c:217:seq_client_alloc_seq()) cli-cli-lustre-MDT0001-osp-MDT0000: Cannot allocate new meta-sequence: rc = -5 [ 5472.559817] LustreError: 167992:0:(fid_request.c:321:seq_client_alloc_fid()) cli-cli-lustre-MDT0001-osp-MDT0000: Can't allocate new sequence: rc = -5 [ 5472.580842] Lustre: lustre-MDT0000: Recovery over after 0:20, of 2 clients 0 recovered and 2 were evicted. [ 5472.996864] Lustre: server umount lustre-MDT0000 complete [ 5473.248922] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5473.274178] LustreError: 166030:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5477.457448] LustreError: 164891:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788473896 with bad export cookie 16442969397041477063 [ 5477.481969] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5483.904180] Lustre: server umount lustre-MDT0001 complete [ 5494.745068] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5497.145906] Key type lgssc unregistered [ 5497.516751] LNet: 169395:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5497.524635] LNetError: 169395:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5497.537656] LNet: Removed LNI 192.168.201.119@tcp [ 5498.326776] Key type .llcrypt unregistered [ 5498.328960] Key type ._llcrypt unregistered [ 5522.429353] Key type ._llcrypt registered [ 5522.431628] Key type .llcrypt registered [ 5522.560113] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5524.079845] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5524.366328] alg: No test for adler32 (adler32-zlib) [ 5525.570953] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 5525.797167] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5527.467367] Key type lgssc registered [ 5528.926386] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5537.581777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5537.621711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5538.958480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5542.979078] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5549.543947] LustreError: 170807:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5549.571697] LustreError: 170807:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5552.285937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5552.755199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5557.287333] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5564.606184] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5569.646576] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5576.096522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5576.571668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5583.287427] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5584.759358] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:225) [ 5590.941375] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5598.176707] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5598.181827] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5598.190606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5599.542955] Lustre: server umount lustre-OST0000 complete [ 5603.808876] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5603.813934] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5603.822540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5605.346965] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5605.348346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5605.379814] Lustre: Skipped 1 previous similar message [ 5610.467359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5610.490562] Lustre: Skipped 1 previous similar message [ 5612.580229] Lustre: server umount lustre-MDT0000 complete [ 5615.584801] LustreError: 171536:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5615.601114] LustreError: 171536:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5615.917883] LustreError: 170788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474035 with bad export cookie 2824421642561392993 [ 5615.924660] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5616.198691] Lustre: server umount lustre-MDT0001 complete [ 5626.169180] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5629.882989] Key type lgssc unregistered [ 5630.392053] LNet: 174336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5630.401845] LNetError: 174336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5630.438131] LNet: Removed LNI 192.168.201.119@tcp [ 5631.670856] Key type .llcrypt unregistered [ 5631.672535] Key type ._llcrypt unregistered [ 5664.315510] Key type ._llcrypt registered [ 5664.317455] Key type .llcrypt registered [ 5664.421139] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5665.372048] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5665.424603] alg: No test for adler32 (adler32-zlib) [ 5666.740572] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 5667.096987] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5668.872907] Key type lgssc registered [ 5670.441220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5679.258145] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5679.286403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5680.765807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5686.227759] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5690.338115] LustreError: 176030:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5690.360993] LustreError: 176030:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5694.611337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5694.936309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5699.423867] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5707.252336] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5714.103196] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5723.536524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5724.254784] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5725.317380] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:257) [ 5732.931680] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5741.530747] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5751.286863] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5751.289791] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5751.305541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5751.327033] LustreError: Skipped 1 previous similar message [ 5755.666459] Lustre: server umount lustre-OST0000 complete [ 5761.505863] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5761.510241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5761.517042] Lustre: Skipped 2 previous similar messages [ 5761.554177] Lustre: Skipped 2 previous similar messages [ 5765.472717] Lustre: server umount lustre-MDT0000 complete [ 5766.633156] LustreError: 176029:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5769.277146] LustreError: 176009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474188 with bad export cookie 9870300571111851964 [ 5769.286302] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5769.288455] LustreError: 176009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5769.686481] Lustre: server umount lustre-MDT0001 complete [ 5781.326867] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5784.555798] Key type lgssc unregistered [ 5784.805374] LNet: 179561:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5784.815935] LNetError: 179561:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5785.829975] LNet: Removed LNI 192.168.201.119@tcp [ 5786.938238] Key type .llcrypt unregistered [ 5786.939529] Key type ._llcrypt unregistered [ 5811.717904] Key type ._llcrypt registered [ 5811.720705] Key type .llcrypt registered [ 5811.853279] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5813.234295] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5813.439523] alg: No test for adler32 (adler32-zlib) [ 5814.593102] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 5814.955727] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5816.688414] Key type lgssc registered [ 5818.037321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5829.936923] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5829.950979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5831.416619] Lustre: *** cfs_fail_loc=135, val=0*** [ 5831.558617] LustreError: 181080:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-2) [ 5831.571637] LustreError: 181080:0:(obd_config.c:2100:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -2 [ 5831.584867] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 5831.584867] [ 5831.605340] LustreError: MGC192.168.201.119@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 5831.624438] LustreError: 181041:0:(tgt_mount.c:1948:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 5831.638095] LustreError: 181041:0:(tgt_mount.c:2734:server_fill_super()) Unable to start targets: -2 [ 5831.645597] LustreError: 181041:0:(tgt_mount.c:2192:server_put_super()) no obd lustre-MDT0000 [ 5831.650344] LustreError: 181041:0:(tgt_mount.c:130:server_deregister_mount()) lustre-MDT0000 not registered [ 5831.725889] Lustre: server umount lustre-MDT0000 complete [ 5831.728737] LustreError: 181041:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 5838.850993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5839.332181] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5843.880292] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5849.568481] LustreError: 181451:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5849.586477] LustreError: 181451:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5851.194677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5851.617792] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5856.755334] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5865.397894] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5871.933095] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5876.385790] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5876.390552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5876.402735] Lustre: Skipped 1 previous similar message [ 5876.424803] Lustre: Skipped 1 previous similar message [ 5880.878932] Lustre: server umount lustre-MDT0000 complete [ 5881.831098] LustreError: 182181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5881.855642] LustreError: 182181:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5884.865550] LustreError: 181431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474304 with bad export cookie 5144927530307140555 [ 5884.882512] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5884.883033] LustreError: 181431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5885.383607] Lustre: server umount lustre-MDT0001 complete [ 5896.534779] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5899.476909] Key type lgssc unregistered [ 5899.904949] LNet: 183774:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5899.921245] LNetError: 183774:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5899.935816] LNet: Removed LNI 192.168.201.119@tcp [ 5900.811683] Key type .llcrypt unregistered [ 5900.818704] Key type ._llcrypt unregistered [ 5925.287827] Key type ._llcrypt registered [ 5925.289649] Key type .llcrypt registered [ 5925.492309] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5925.807625] Key type .llcrypt unregistered [ 5925.810421] Key type ._llcrypt unregistered [ 5945.742874] Key type ._llcrypt registered [ 5945.744930] Key type .llcrypt registered [ 5945.954713] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5948.689858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5948.789128] alg: No test for adler32 (adler32-zlib) [ 5950.180567] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 5950.707764] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5952.599179] Key type lgssc registered [ 5953.918798] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5961.683606] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5961.703521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5962.785889] Lustre: 185912:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 5978.271474] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5983.391969] LustreError: 185899:0:(mgc_request.c:317:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 5988.319471] Lustre: 185914:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 5998.831588] Lustre: 185912:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6004.199463] Lustre: 185914:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6009.119261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6009.343016] Lustre: 185912:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6017.548451] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6019.553959] Lustre: 185912:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6019.578886] Lustre: 185912:0:(ldlm_lib.c:1248:target_handle_connect()) Skipped 1 previous similar message [ 6026.163098] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6034.180885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6035.104398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6035.200384] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:259 to 0x280000401:289) [ 6040.469432] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6040.545608] LustreError: 185930:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6040.569693] LustreError: 185930:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6044.653081] LustreError: 185929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6044.696281] LustreError: 185929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6049.767508] LustreError: 186942:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6049.805589] LustreError: 186942:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6050.384868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6050.780074] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6056.324363] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6064.645696] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6070.542972] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6073.854236] Lustre: Setting parameter lustre-OST0000.ost.client_cache_seconds=115 in log lustre-OST0000 [ 6086.114788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6086.123138] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6086.145464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6086.625083] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6086.672337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6089.577938] Lustre: server umount lustre-OST0000 complete [ 6096.865080] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6096.873383] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6096.892091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6096.896569] Lustre: Skipped 1 previous similar message [ 6099.328363] Lustre: server umount lustre-MDT0000 complete [ 6101.988159] LustreError: 188276:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6102.003291] LustreError: 188276:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6102.558339] LustreError: 187683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474521 with bad export cookie 13461413355133735082 [ 6102.568465] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6102.579421] LustreError: 187683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6103.001463] Lustre: server umount lustre-MDT0001 complete [ 6114.970650] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 6117.977315] Key type lgssc unregistered [ 6118.152573] LNet: 189821:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6118.159738] LNetError: 189821:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6118.180874] LNet: Removed LNI 192.168.201.119@tcp [ 6119.264991] Key type .llcrypt unregistered [ 6119.266659] Key type ._llcrypt unregistered [ 6150.425835] Key type ._llcrypt registered [ 6150.427714] Key type .llcrypt registered [ 6150.611271] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 6153.050749] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6153.346910] alg: No test for adler32 (adler32-zlib) [ 6154.991511] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 6155.330742] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 6157.208089] Key type lgssc registered [ 6158.591227] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6166.434973] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6166.461528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6167.971311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6172.479297] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6178.284431] LustreError: 191230:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6178.304695] LustreError: 191230:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6180.946751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6181.255140] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6187.560157] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6196.638172] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6202.618270] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6211.289046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6211.805683] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6212.884329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:259 to 0x280000401:321) [ 6220.152682] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6228.293771] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6234.266562] Lustre: Failing over lustre-MDT0000 [ 6234.539075] Lustre: server umount lustre-MDT0000 complete [ 6236.156600] LustreError: 193735:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6236.644908] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6236.669966] Lustre: Skipped 1 previous similar message [ 6241.225704] LustreError: 191227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6241.249189] LustreError: 191227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6241.766832] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6241.796750] Lustre: 193980:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6246.346182] LustreError: 192162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6246.370506] LustreError: 192162:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6246.883483] Lustre: 193978:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6251.478113] LustreError: 191226:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6251.517873] LustreError: 191226:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6252.019180] Lustre: 193978:0:(ldlm_lib.c:1248:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 6256.181666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6256.582305] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.19@tcp (not set up) [ 6256.756139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6256.804738] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 6261.308366] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6261.699511] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 6262.266070] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6262.323638] Lustre: lustre-MDT0000: Recovery over after 0:01, of 2 clients 2 recovered and 0 were evicted. [ 6262.400559] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:259 to 0x280000401:353) [ 6266.374948] Lustre: Setting parameter lustre-MDT0000.mdt.identity_acquire_expire=35 in log lustre-MDT0000 [ 6274.144363] Lustre: Setting parameter lustre-MDT0000-mdc.mdc.max_rpcs_in_flight=13 in log lustre-client [ 6282.720472] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6282.742985] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6282.759161] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6282.766190] Lustre: Skipped 2 previous similar messages [ 6282.780143] Lustre: Skipped 1 previous similar message [ 6287.845221] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6287.862047] Lustre: Skipped 1 previous similar message [ 6288.356727] Lustre: server umount lustre-OST0000 complete [ 6292.960190] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6292.970982] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6292.971212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6293.006927] Lustre: Skipped 1 previous similar message [ 6298.087312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6298.092447] Lustre: Skipped 2 previous similar messages [ 6298.961900] Lustre: server umount lustre-MDT0000 complete [ 6303.113498] LustreError: 191212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474722 with bad export cookie 10455293284093209916 [ 6303.116939] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6303.123588] LustreError: 191212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6303.350847] Lustre: server umount lustre-MDT0001 complete [ 6320.410904] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 6323.180316] Key type lgssc unregistered [ 6323.460367] LNet: 196142:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6323.469977] LNetError: 196142:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6323.486252] LNet: Removed LNI 192.168.201.119@tcp [ 6324.476151] Key type .llcrypt unregistered [ 6324.476995] Key type ._llcrypt unregistered [ 6353.368545] Key type ._llcrypt registered [ 6353.375605] Key type .llcrypt registered [ 6353.582853] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 6355.234643] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6355.325402] alg: No test for adler32 (adler32-zlib) [ 6356.497714] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 6356.727575] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 6358.399277] Key type lgssc registered [ 6360.335663] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6372.262933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6372.279273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6374.037728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6378.621885] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6384.610069] LustreError: 197552:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6384.634662] LustreError: 197552:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6387.251847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6387.992663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6393.619031] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6401.815645] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6408.328907] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6415.299142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6415.802733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6418.898843] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:385) [ 6422.337714] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6429.342620] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6432.289679] Lustre: Setting parameter lustre-client.llite.max_read_ahead_whole_mb=5 in log lustre-client [ 6441.161459] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=6 in log lustre-client [ 6458.521513] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 6473.190989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6473.193124] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6473.210259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6473.222618] Lustre: Skipped 1 previous similar message [ 6478.075019] Lustre: server umount lustre-OST0000 complete [ 6483.428155] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6483.439174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6483.445899] Lustre: Skipped 2 previous similar messages [ 6483.468334] Lustre: Skipped 1 previous similar message [ 6487.699105] Lustre: server umount lustre-MDT0000 complete [ 6488.543981] LustreError: 200155:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6488.557659] LustreError: 200155:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6490.808268] LustreError: 197532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788474909 with bad export cookie 7910421963281486867 [ 6490.809757] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6490.814177] LustreError: 197532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6491.239528] Lustre: server umount lustre-MDT0001 complete [ 6502.382851] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 6505.720745] Key type lgssc unregistered [ 6506.035084] LNet: 201446:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6506.044022] LNetError: 201446:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6506.068228] LNet: Removed LNI 192.168.201.119@tcp [ 6507.015450] Key type .llcrypt unregistered [ 6507.017186] Key type ._llcrypt unregistered [ 6526.914167] Key type ._llcrypt registered [ 6526.916736] Key type .llcrypt registered [ 6527.171097] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 6528.613936] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6528.656154] alg: No test for adler32 (adler32-zlib) [ 6529.813097] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 6530.055204] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 6531.751482] Key type lgssc registered [ 6533.465594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6543.968684] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6544.005881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6545.716884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6550.534463] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6555.106933] LustreError: 202706:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6555.124130] LustreError: 202706:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6560.227410] LustreError: 202707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6560.806317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6561.253744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6566.831812] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6574.785378] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6581.581209] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6589.711501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6589.952765] LustreError: 204527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6590.232799] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6595.604047] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:417) [ 6598.377779] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6607.054190] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6613.400713] Lustre: Modifying parameter lustre-OST0000.ost.client_cache_seconds=230 in log lustre-OST0000 [ 6624.446508] Lustre: Modifying parameter lustre-OST0000.ost.client_cache_seconds=115 in log lustre-OST0000 [ 6630.155643] Lustre: Setting parameter lustre-OST0000.osd.auto_scrub=1 in log lustre-OST0000 [ 6634.977590] Lustre: Modifying parameter lustre-OST0000.osd.auto_scrub=2592000 in log lustre-OST0000 [ 6646.712726] Lustre: server umount lustre-OST0000 complete [ 6652.898241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6652.902100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6652.919914] Lustre: Skipped 1 previous similar message [ 6652.943211] Lustre: Skipped 1 previous similar message [ 6658.017285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6661.333917] Lustre: server umount lustre-MDT0000 complete [ 6663.135095] Lustre: 202017:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788475066/real 1788475066] req@00000000989be309 x1875351908190592/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788475082 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6663.150908] LustreError: 202706:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6663.166823] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6663.206672] LustreError: 202706:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6664.821470] LustreError: 202687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788475084 with bad export cookie 996188136833663057 [ 6664.823316] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6664.831581] LustreError: 202687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6665.276222] Lustre: server umount lustre-MDT0001 complete [ 6677.360163] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 6680.620738] Key type lgssc unregistered [ 6680.969987] LNet: 207389:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6680.978137] LNetError: 207389:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6681.026153] LNet: Removed LNI 192.168.201.119@tcp [ 6681.934311] Key type .llcrypt unregistered [ 6681.942537] Key type ._llcrypt unregistered [ 6709.342644] Key type ._llcrypt registered [ 6709.352271] Key type .llcrypt registered [ 6709.471790] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 6710.554386] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6710.646292] alg: No test for adler32 (adler32-zlib) [ 6711.912748] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 6712.229731] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 6713.928315] Key type lgssc registered [ 6715.381983] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6724.082801] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6724.128717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6725.689064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6731.527391] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6734.817657] LustreError: 208797:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6734.854870] LustreError: 208797:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6739.729789] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6739.937180] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 6740.148935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6745.335949] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6753.010936] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6759.147987] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6766.016763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6766.413855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6770.570585] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:449) [ 6772.637258] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6780.546696] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6800.355787] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6800.358861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6800.378657] Lustre: Skipped 1 previous similar message [ 6800.405994] Lustre: Skipped 1 previous similar message [ 6803.777683] Lustre: server umount lustre-OST0000 complete [ 6810.592623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6810.610584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6815.712570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6815.714892] Lustre: Skipped 2 previous similar messages [ 6820.834096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6820.852416] Lustre: Skipped 1 previous similar message [ 6822.367259] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6822.554688] Lustre: server umount lustre-MDT0000 complete [ 6825.957227] LustreError: 208797:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6826.548431] LustreError: 211290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788475245 with bad export cookie 12078322806672421270 [ 6826.551726] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6826.574492] LustreError: 211290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6827.022767] Lustre: server umount lustre-MDT0001 complete [ 6839.436909] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 6842.754814] Key type lgssc unregistered [ 6843.029239] LNet: 212482:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6843.041084] LNetError: 212482:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6844.070317] LNet: Removed LNI 192.168.201.119@tcp [ 6844.947629] Key type .llcrypt unregistered [ 6844.949633] Key type ._llcrypt unregistered [ 6865.571994] Key type ._llcrypt registered [ 6865.578662] Key type .llcrypt registered [ 6865.769804] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 6867.679185] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6867.848919] alg: No test for adler32 (adler32-zlib) [ 6869.065492] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 6869.414203] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 6871.135377] Key type lgssc registered [ 6872.842935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6882.094326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6882.127135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6883.806512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6889.804981] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6893.028610] LustreError: 213743:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6893.058495] LustreError: 213743:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6898.155496] LustreError: 213744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6900.540608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6901.248718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6904.820350] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6913.473902] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6921.363942] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6931.648639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6932.376640] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6940.621463] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 6948.800397] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6952.892539] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:481) [ 6973.411344] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6973.416419] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6973.445195] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6978.533707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6978.543769] Lustre: Skipped 1 previous similar message [ 6979.475936] Lustre: server umount lustre-OST0000 complete [ 6983.649320] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6983.668737] Lustre: Skipped 1 previous similar message [ 6983.678067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6983.686395] Lustre: Skipped 1 previous similar message [ 6988.257242] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6988.273896] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6988.319470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6989.948194] Lustre: server umount lustre-MDT0000 complete [ 6993.894216] LustreError: 213744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6993.934962] LustreError: 213744:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6996.099826] LustreError: 213724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788475415 with bad export cookie 3284708204226915395 [ 6996.114054] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6996.515713] Lustre: server umount lustre-MDT0001 complete [ 7009.657805] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 7013.320437] Key type lgssc unregistered [ 7013.782942] LNet: 217454:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7013.791021] LNetError: 217454:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7013.808762] LNet: Removed LNI 192.168.201.119@tcp [ 7014.996988] Key type .llcrypt unregistered [ 7014.999201] Key type ._llcrypt unregistered [ 7034.759563] Key type ._llcrypt registered [ 7034.761948] Key type .llcrypt registered [ 7034.978228] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 7036.885833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7036.985326] alg: No test for adler32 (adler32-zlib) [ 7038.388284] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 7038.699645] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 7040.480049] Key type lgssc registered [ 7042.421806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7052.797504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7052.834500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7054.333848] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7059.241925] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7064.554176] LustreError: 218718:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7064.572861] LustreError: 218718:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7065.582097] LustreError: 218719:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7069.723500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7070.252970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7076.320543] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7085.635894] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7092.221642] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7099.922287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7100.373439] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7106.860954] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7114.114883] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7125.045034] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:513) [ 7134.453165] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7134.533094] Lustre: MGS: Regenerating lustre-OST0001 log by user request: rc = 0 [ 7134.786358] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7142.277220] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7152.889936] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7154.842051] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7167.410976] Lustre: Permanently deactivating lustre-OST0001 [ 7167.459671] Lustre: Setting parameter lustre-OST0001-osc.osc.active=0 in log lustre-client [ 7177.094907] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 7186.393113] Lustre: Permanently reactivating lustre-OST0001 [ 7186.419283] Lustre: Modifying parameter lustre-OST0001-osc.osc.active=1 in log lustre-client [ 7186.424714] Lustre: Skipped 2 previous similar messages [ 7187.061321] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7187.078943] Lustre: lustre-OST0001: Client lustre-MDT0001-mdtlov_UUID (at 0@lo) reconnecting [ 7187.085771] LustreError: lustre-OST0001-osc-MDT0001: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 7187.096978] Lustre: lustre-OST0001-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7187.151907] LustreError: lustre-OST0001-osc-MDT0000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 7202.789850] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7202.790073] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7202.790082] Lustre: Skipped 1 previous similar message [ 7202.790537] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7202.823682] LustreError: Skipped 1 previous similar message [ 7204.322673] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7204.344347] Lustre: Skipped 2 previous similar messages [ 7209.447241] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7209.473736] Lustre: Skipped 1 previous similar message [ 7214.047836] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7214.156268] Lustre: server umount lustre-OST0001 complete [ 7214.600968] LustreError: 220540:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7214.627050] LustreError: 220540:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7219.684967] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7219.708368] LustreError: 221788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7219.716409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7219.725555] Lustre: Skipped 2 previous similar messages [ 7219.772409] LustreError: 221788:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7224.810379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7224.811259] LustreError: 220538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7224.817550] Lustre: Skipped 2 previous similar messages [ 7224.848574] LustreError: 220538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7225.889986] Lustre: server umount lustre-OST0000 complete [ 7234.016081] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7234.023926] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7234.042651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7238.122692] Lustre: server umount lustre-MDT0000 complete [ 7240.163737] LustreError: 219446:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7240.184360] LustreError: 219446:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7243.053668] LustreError: 218698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788475662 with bad export cookie 7562509930169600315 [ 7243.056681] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7243.062193] LustreError: 218698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7243.521851] Lustre: server umount lustre-MDT0001 complete [ 7256.320648] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 7259.744712] Key type lgssc unregistered [ 7260.041846] LNet: 223823:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7260.053031] LNetError: 223823:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7260.070171] LNet: Removed LNI 192.168.201.119@tcp [ 7260.735157] Key type .llcrypt unregistered [ 7260.737097] Key type ._llcrypt unregistered [ 7308.412314] Key type ._llcrypt registered [ 7308.415134] Key type .llcrypt registered [ 7308.514618] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 7309.761872] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7309.802686] alg: No test for adler32 (adler32-zlib) [ 7310.830597] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 7311.037146] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 7312.655164] Key type lgssc registered [ 7313.816392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7324.008365] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7324.064311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7325.628637] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 7325.674248] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7326.026325] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.420292] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7347.094084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7347.370718] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7347.445732] Lustre: 226026:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7347.783247] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7354.100486] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7362.903469] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7369.450572] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7376.383805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7376.701369] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 7376.999897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7383.509876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:545) [ 7386.063895] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7396.249303] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7399.572512] Lustre: Setting parameter lustre-client.llite.max_read_ahead_whole_mb=1 in log lustre-client [ 7407.469980] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=2 in log lustre-client [ 7413.898890] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=3 in log lustre-client [ 7421.782793] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 7431.081799] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=5 in log lustre-client [ 7447.592286] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=3 in log lustre-client [ 7447.597636] Lustre: Skipped 1 previous similar message [ 7480.969165] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=3 in log lustre-client [ 7480.981735] Lustre: Skipped 3 previous similar messages [ 7547.884243] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 7547.898186] Lustre: Skipped 8 previous similar messages [ 7572.960444] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7572.965522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7572.983876] LustreError: Skipped 1 previous similar message [ 7572.987854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7573.036796] Lustre: Skipped 1 previous similar message [ 7576.097795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7576.106139] Lustre: Skipped 1 previous similar message [ 7579.142332] Lustre: server umount lustre-OST0000 complete [ 7586.273181] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7586.278614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7586.310681] Lustre: Skipped 1 previous similar message [ 7586.351754] Lustre: Skipped 2 previous similar messages [ 7591.702442] Lustre: server umount lustre-MDT0000 complete [ 7596.288215] LustreError: 228940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788476015 with bad export cookie 14580564528862054247 [ 7596.293121] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7596.302542] LustreError: 228940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7596.564505] Lustre: server umount lustre-MDT0001 complete [ 7608.504822] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 7611.326409] Key type lgssc unregistered [ 7611.629482] LNet: 230830:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7611.642062] LNetError: 230830:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7611.659059] LNet: Removed LNI 192.168.201.119@tcp [ 7612.590417] Key type .llcrypt unregistered [ 7612.593769] Key type ._llcrypt unregistered [ 7637.984210] Key type ._llcrypt registered [ 7637.987699] Key type .llcrypt registered [ 7638.147642] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 7639.879745] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7639.929051] alg: No test for adler32 (adler32-zlib) [ 7641.177169] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 7641.588993] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 7643.319260] Key type lgssc registered [ 7644.961917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7655.218707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7655.241629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7656.898809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7661.937613] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7667.170152] LustreError: 232259:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7667.199552] LustreError: 232259:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7670.240247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7670.680788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7674.896795] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7682.133594] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7687.364801] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7695.422767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7695.771321] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7701.942601] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:355 to 0x280000401:577) [ 7704.546676] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7712.600868] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7732.709961] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7732.724960] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7732.753452] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7733.571271] Lustre: server umount lustre-OST0000 complete [ 7740.897188] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7740.901546] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7740.911896] Lustre: Skipped 1 previous similar message [ 7743.914665] Lustre: server umount lustre-MDT0000 complete [ 7746.016127] LustreError: 234749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7746.045438] LustreError: 234749:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7748.043894] LustreError: 234840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788476167 with bad export cookie 3766887087426721620 [ 7748.048338] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7748.058888] LustreError: 234840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7758.737802] Lustre: server umount lustre-MDT0001 complete [ 7772.998287] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 7776.926161] Key type lgssc unregistered [ 7777.370661] LNet: 235951:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7777.377989] LNetError: 235951:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7777.400205] LNet: Removed LNI 192.168.201.119@tcp [ 7778.615855] Key type .llcrypt unregistered [ 7778.617902] Key type ._llcrypt unregistered [ 7804.178792] Key type ._llcrypt registered [ 7804.184851] Key type .llcrypt registered [ 7804.283495] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 7805.537661] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7805.548919] alg: No test for adler32 (adler32-zlib) [ 7806.681828] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 7807.101873] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 7808.863261] Key type lgssc registered [ 7810.338567] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8081.302662] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 8084.245915] Key type lgssc unregistered [ 8084.568742] LNet: 237983:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8084.576679] LNetError: 237983:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8084.605049] LNet: Removed LNI 192.168.201.119@tcp [ 8085.535713] Key type .llcrypt unregistered [ 8085.544285] Key type ._llcrypt unregistered [ 8115.583202] Key type ._llcrypt registered [ 8115.588470] Key type .llcrypt registered [ 8115.765724] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 8132.903658] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8134.684830] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8134.762944] alg: No test for adler32 (adler32-zlib) [ 8135.822230] Lustre: Lustre: Build Version: 2.17.58_4_g0efcf2a [ 8136.109388] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 8137.863186] Key type lgssc registered [ 8139.221893] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8172.208718] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7918 sec ========== 19:03:10 (1788476590) [ 8174.084832] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 19:03:11 (1788476591) === [ 8177.479873] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 19:03:15 (1788476595) === [ 8217.909590] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 8220.923389] Key type lgssc unregistered [ 8221.233736] LNet: 243219:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8221.237419] LNetError: 243219:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8221.245613] LNet: Removed LNI 192.168.201.119@tcp [ 8222.167489] Key type .llcrypt unregistered [ 8222.169203] Key type ._llcrypt unregistered