[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 481569491 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003348] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009015] pid_max: default: 32768 minimum: 301 [ 0.010143] LSM: Security Framework initializing [ 0.011065] Yama: becoming mindful. [ 0.012047] SELinux: Initializing. [ 0.013078] *** VALIDATE selinux *** [ 0.021189] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026037] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028040] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030069] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032106] *** VALIDATE tmpfs *** [ 0.033483] *** VALIDATE proc *** [ 0.034246] *** VALIDATE cgroup *** [ 0.035010] *** VALIDATE cgroup2 *** [ 0.037260] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038157] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040028] Spectre V2 : User space: Vulnerable [ 0.042009] Speculative Store Bypass: Vulnerable [ 0.044852] debug: unmapping init [mem 0xffffffff8a259000-0xffffffff8a260fff] [ 0.046853] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047697] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048028] ... version: 2 [ 0.049013] ... bit width: 48 [ 0.050019] ... generic registers: 4 [ 0.051014] ... value mask: 0000ffffffffffff [ 0.052015] ... max period: 00007fffffffffff [ 0.053015] ... fixed-purpose events: 3 [ 0.054013] ... event mask: 000000070000000f [ 0.056222] rcu: Hierarchical SRCU implementation. [ 0.058453] smp: Bringing up secondary CPUs ... [ 0.059602] x86: Booting SMP configuration: [ 0.060026] .... node #0, CPUs: #1 #2 #3 [ 0.063452] smp: Brought up 1 node, 4 CPUs [ 0.065015] smpboot: Max logical packages: 1 [ 0.066021] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.220623] node 0 deferred pages initialised in 151ms [ 0.222106] devtmpfs: initialized [ 0.224285] x86/mm: Memory block size: 128MB [ 0.227901] gcov: version magic: 0x41383552 [ 0.229313] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.230086] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.231276] pinctrl core: initialized pinctrl subsystem [ 0.232179] [ 0.232773] ************************************************************* [ 0.233020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.234014] ** ** [ 0.235021] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.236015] ** ** [ 0.237018] ** This means that this kernel is built to expose internal ** [ 0.238014] ** IOMMU data structures, which may compromise security on ** [ 0.239025] ** your system. ** [ 0.240018] ** ** [ 0.241013] ** If you see this message and you are not debugging the ** [ 0.242018] ** kernel, report this immediately to your vendor! ** [ 0.243015] ** ** [ 0.244014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.245018] ************************************************************* [ 0.246677] NET: Registered protocol family 16 [ 0.247460] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.248066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.249081] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.250430] cpuidle: using governor menu [ 0.253244] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.256669] PCI: Using configuration type 1 for base access [ 0.259156] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.271145] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.272044] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.274045] cryptd: max_cpu_qlen set to 1000 [ 0.277080] ACPI: Added _OSI(Module Device) [ 0.278015] ACPI: Added _OSI(Processor Device) [ 0.278967] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.279018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.283222] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.287525] ACPI: Interpreter enabled [ 0.288078] ACPI: PM: (supports S0 S3 S4 S5) [ 0.289014] ACPI: Using IOAPIC for interrupt routing [ 0.290104] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.291391] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.300666] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.301040] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.302021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.303086] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.305234] acpiphp: Slot [2] registered [ 0.306160] acpiphp: Slot [5] registered [ 0.307193] acpiphp: Slot [6] registered [ 0.308122] acpiphp: Slot [7] registered [ 0.309139] acpiphp: Slot [8] registered [ 0.310142] acpiphp: Slot [9] registered [ 0.311073] acpiphp: Slot [10] registered [ 0.312129] acpiphp: Slot [3] registered [ 0.313127] acpiphp: Slot [4] registered [ 0.314110] acpiphp: Slot [11] registered [ 0.315134] acpiphp: Slot [12] registered [ 0.316092] acpiphp: Slot [13] registered [ 0.317113] acpiphp: Slot [14] registered [ 0.318105] acpiphp: Slot [15] registered [ 0.319152] acpiphp: Slot [16] registered [ 0.320126] acpiphp: Slot [17] registered [ 0.321114] acpiphp: Slot [18] registered [ 0.322128] acpiphp: Slot [19] registered [ 0.323123] acpiphp: Slot [20] registered [ 0.324114] acpiphp: Slot [21] registered [ 0.325104] acpiphp: Slot [22] registered [ 0.326147] acpiphp: Slot [23] registered [ 0.327122] acpiphp: Slot [24] registered [ 0.328115] acpiphp: Slot [25] registered [ 0.329109] acpiphp: Slot [26] registered [ 0.330113] acpiphp: Slot [27] registered [ 0.331092] acpiphp: Slot [28] registered [ 0.332143] acpiphp: Slot [29] registered [ 0.333116] acpiphp: Slot [30] registered [ 0.334120] acpiphp: Slot [31] registered [ 0.335071] PCI host bridge to bus 0000:00 [ 0.336023] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.337051] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.338031] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.339026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.340027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.341030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.342196] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.343967] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.345380] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.351018] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.354565] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.355023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.356026] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.357031] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.358782] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.360094] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.361050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.363017] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.365911] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.372017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.375033] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.379113] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.382019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.385021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.394031] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.400993] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.404028] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.407018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.414032] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.419930] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.422020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.426022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.433022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.439470] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.443024] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.446019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.454022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.459903] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.482021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.505021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.540040] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.556249] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.570029] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.581018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.600018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.618781] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.621614] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.625087] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.629434] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.632281] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.636245] iommu: Default domain type: Passthrough [ 0.638578] SCSI subsystem initialized [ 0.640178] ACPI: bus type USB registered [ 0.642113] usbcore: registered new interface driver usbfs [ 0.644112] usbcore: registered new interface driver hub [ 0.645098] usbcore: registered new device driver usb [ 0.647259] pps_core: LinuxPPS API ver. 1 registered [ 0.650015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.654064] PTP clock support registered [ 0.656181] EDAC MC: Ver: 3.0.0 [ 0.659130] PCI: Using ACPI for IRQ routing [ 0.661528] NetLabel: Initializing [ 0.663018] NetLabel: domain hash size = 128 [ 0.664012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.666099] NetLabel: unlabeled traffic allowed by default [ 0.670290] vgaarb: loaded [ 0.672375] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.674016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.682000] clocksource: Switched to clocksource kvm-clock [ 0.805190] VFS: Disk quotas dquot_6.6.0 [ 0.806762] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.809196] *** VALIDATE ramfs *** [ 0.810475] *** VALIDATE hugetlbfs *** [ 0.812048] pnp: PnP ACPI init [ 0.814397] pnp: PnP ACPI: found 6 devices [ 0.832429] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.835570] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.837662] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.839673] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.841915] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.844644] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.847881] NET: Registered protocol family 2 [ 0.850205] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.854881] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.858307] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.864356] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.867889] TCP: Hash tables configured (established 65536 bind 65536) [ 0.870126] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.872696] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.875090] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.878490] NET: Registered protocol family 1 [ 0.881324] RPC: Registered named UNIX socket transport module. [ 0.883848] RPC: Registered udp transport module. [ 0.886053] RPC: Registered tcp transport module. [ 0.887898] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.890525] NET: Registered protocol family 44 [ 0.891893] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.893626] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.895709] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.898268] PCI: CLS 0 bytes, default 64 [ 0.900157] Unpacking initramfs... [ 2.309940] debug: unmapping init [mem 0xffff88effcc54000-0xffff88effffbffff] [ 2.313042] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.315278] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.318645] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.787846] Initialise system trusted keyrings [ 2.789628] Key type blacklist registered [ 2.791597] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.799542] zbud: loaded [ 2.801974] *** VALIDATE nfs *** [ 2.802904] *** VALIDATE nfs4 *** [ 2.804432] pstore: using deflate compression [ 2.808247] Platform Keyring initialized [ 2.907551] NET: Registered protocol family 38 [ 2.909857] Key type asymmetric registered [ 2.911643] Asymmetric key parser 'x509' registered [ 2.914198] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.918149] io scheduler mq-deadline registered [ 2.922131] io scheduler kyber registered [ 2.924244] io scheduler bfq registered [ 2.926477] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.929759] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.933848] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.936970] ACPI: Power Button [PWRF] [ 2.942599] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.951535] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.965310] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.971797] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.993169] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.021776] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.051861] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.057167] Non-volatile memory driver v1.3 [ 3.059161] Linux agpgart interface v0.103 [ 3.094692] virtio_blk virtio1: [vda] 149768 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.097854] vda: detected capacity change from 0 to 76681216 [ 3.133524] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.136775] vdb: detected capacity change from 0 to 1073741824 [ 3.153856] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.156670] vdc: detected capacity change from 0 to 2621440000 [ 3.171660] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.175084] vdd: detected capacity change from 0 to 2621440000 [ 3.196498] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.199412] vde: detected capacity change from 0 to 4294967296 [ 3.219666] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.222892] vdf: detected capacity change from 0 to 4294967296 [ 3.236450] libphy: Fixed MDIO Bus: probed [ 3.252196] usbcore: registered new interface driver usbserial_generic [ 3.255413] usbserial: USB Serial support registered for generic [ 3.258422] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.264513] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.266193] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.268876] mousedev: PS/2 mouse device common for all mice [ 3.273593] rtc_cmos 00:05: RTC can wake from S4 [ 3.275710] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.280392] rtc_cmos 00:05: registered as rtc0 [ 3.291977] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.293337] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.297979] intel_pstate: CPU model not supported [ 3.309114] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.311741] hid: raw HID events driver (C) Jiri Kosina [ 3.317930] usbcore: registered new interface driver usbhid [ 3.320439] usbhid: USB HID core driver [ 3.322502] drop_monitor: Initializing network drop monitor service [ 3.325410] Initializing XFRM netlink socket [ 3.327827] NET: Registered protocol family 10 [ 3.331516] Segment Routing with IPv6 [ 3.333192] NET: Registered protocol family 17 [ 3.334909] mpls_gso: MPLS GSO support [ 3.343352] RAS: Correctable Errors collector initialized. [ 3.345845] AVX version of gcm_enc/dec engaged. [ 3.347793] AES CTR mode by8 optimization enabled [ 3.714579] sched_clock: Marking stable (3714554243, 0)->(4899501437, -1184947194) [ 3.752319] registered taskstats version 1 [ 3.757424] Loading compiled-in X.509 certificates [ 3.760765] zswap: loaded using pool lzo/zbud [ 3.912693] Key type big_key registered [ 3.931997] Key type encrypted registered [ 3.934806] ima: No TPM chip found, activating TPM-bypass! [ 3.943602] ima: Allocated hash algorithm: sha1 [ 3.945454] ima: No architecture policies found [ 3.950365] evm: Initialising EVM extended attributes: [ 3.957077] evm: security.selinux [ 3.959634] evm: security.ima [ 3.961776] evm: security.capability [ 3.963687] evm: HMAC attrs: 0x1 [ 3.980723] rtc_cmos 00:05: setting system clock to 2026-09-06 08:35:11 UTC (1788683711) [ 3.996464] debug: unmapping init [mem 0xffffffff8b203000-0xffffffff8b3fffff] [ 4.002181] debug: unmapping init [mem 0xffffffff89f82000-0xffffffff8a258fff] [ 4.027146] Write protecting the kernel read-only data: 28672k [ 4.039229] debug: unmapping init [mem 0xffffffff88603000-0xffffffff887fffff] [ 4.049409] debug: unmapping init [mem 0xffffffff88f14000-0xffffffff88ffffff] [ 4.203278] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.242630] systemd[1]: Detected virtualization kvm. [ 4.244767] systemd[1]: Detected architecture x86-64. [ 4.246723] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.443849] systemd[1]: No hostname configured. [ 4.448071] systemd[1]: Set hostname to . [ 4.450183] random: systemd: uninitialized urandom read (16 bytes read) [ 4.457971] systemd[1]: Initializing machine ID from random generator. [ 4.707813] random: ln: uninitialized urandom read (6 bytes read) [ 5.019949] random: systemd: uninitialized urandom read (16 bytes read) [ 5.031525] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 5.046511] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 5.063659] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.875671] device-mapper: uevent: version 1.0.3 [ 6.877921] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 8.993878] virtio_net virtio0 ens2: renamed from eth0 [ 9.066710] random: fast init done [ 9.625166] scsi host0: ata_piix [ 9.803342] scsi host1: ata_piix [ 9.805110] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 9.808243] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 14.498341] random: crng init done [ 14.502807] random: 7 urandom warning(s) missed due to ratelimiting [ 16.194241] dracut-initqueue[585]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 18.548186] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.844350] printk: systemd: 26 output lines suppressed due to ratelimiting [ 22.912270] SELinux: Disabled at runtime. [ 23.064372] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 23.099934] systemd[1]: Detected virtualization kvm. [ 23.110316] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 24.606681] systemd[1]: initrd-switch-root.service: Succeeded. [ 24.613404] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 24.624543] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 24.639607] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 24.651543] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 24.667728] systemd[1]: Starting Journal Service... Starting Journal Service... [ 24.705557] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... Starting Remount Root and Kernel File Systems... Starting udev Coldplug all Devices... [ 25.325490] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 26.437553] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.739108] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 27.744346] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.329327] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 28.410455] EDAC sbridge: Ver: 1.1.2 [ 32.150741] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 33.117425] NFS: Registering the id_resolver key type [ 33.119832] Key type id_resolver registered [ 33.121846] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started Daily Cleanup of Temporary Directories. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg118-server login: [ 95.212930] libcfs: loading out-of-tree module taints kernel. [ 95.243840] Key type ._llcrypt registered [ 95.246046] Key type .llcrypt registered [ 95.352875] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 116.849774] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 118.755833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 118.785791] alg: No test for adler32 (adler32-zlib) [ 120.512950] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 121.555933] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 123.359185] Key type lgssc registered [ 125.707503] Lustre: Echo OBD driver; http://www.lustre.org/ [ 144.477824] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 184.412093] hrtimer: interrupt took 3085301 ns [ 187.722190] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 203.091194] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 203.133472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 204.475432] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 204.587844] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 204.699274] Lustre: lustre-MDT0000: new disk, initializing [ 204.806386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 204.845045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 210.704631] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.444895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 225.657149] Lustre: 6520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 225.720345] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 225.725355] Lustre: Skipped 1 previous similar message [ 225.851870] Lustre: lustre-MDT0001: new disk, initializing [ 225.939674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 225.991272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 226.006834] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 230.301499] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.155580] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 248.694782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 249.153311] Lustre: lustre-OST0000: new disk, initializing [ 249.156113] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 249.163651] Lustre: 8458:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 249.256776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 256.078857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 256.090951] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.252682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 256.460982] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 274.125404] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 274.223919] Lustre: lustre-OST0001: new disk, initializing [ 274.226429] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 274.229904] Lustre: 9531:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 274.283317] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 282.130740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 282.138662] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 282.202342] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 282.237338] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 295.546320] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 302.134572] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 308.562526] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing check_logdir /tmp/testlogs/ [ 313.871360] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing yml_node [ 319.567519] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 322.605344] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 325.569345] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 327.705577] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Sep 6 04:40:33 EDT 2026 [ 346.959342] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 348.334155] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 364.001651] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 364.004944] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 364.032109] Lustre: Skipped 1 previous similar message [ 364.055839] Lustre: Skipped 3 previous similar messages [ 369.123587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 369.136825] Lustre: Skipped 3 previous similar messages [ 369.681995] Lustre: server umount lustre-MDT0000 complete [ 373.638701] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684081 with bad export cookie 12419740506181694258 [ 373.647672] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 373.657396] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 374.261677] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.272624] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 374.279378] Lustre: Skipped 3 previous similar messages [ 374.313098] Lustre: Skipped 1 previous similar message [ 378.273804] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 378.279208] Lustre: Skipped 1 previous similar message [ 380.008071] Lustre: server umount lustre-MDT0001 complete [ 384.973924] Lustre: server umount lustre-OST0000 complete [ 389.412141] Lustre: server umount lustre-OST0001 complete [ 397.557827] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 409.734310] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 447.517386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 447.753337] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 447.788902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 447.889670] Lustre: lustre-MDT0000: new disk, initializing [ 447.992280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 448.006702] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 452.846498] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 464.880631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 465.015792] Lustre: 15084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 465.082644] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 465.086230] Lustre: Skipped 1 previous similar message [ 465.155512] Lustre: lustre-MDT0001: new disk, initializing [ 465.278807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 465.318367] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 465.336945] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 470.527362] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 480.734254] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 486.974711] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 495.489186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 495.924420] Lustre: lustre-OST0000: new disk, initializing [ 495.936733] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 495.962657] Lustre: 17055:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 496.095864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 498.144672] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 498.156483] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 498.356108] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 503.277454] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 515.475520] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 521.051220] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 521.382938] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 526.345086] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 526.624378] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 528.865198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 528.867912] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 528.869686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 528.871348] Lustre: Skipped 1 previous similar message [ 539.124474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 539.147471] Lustre: Skipped 3 previous similar messages [ 543.199169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 543.407424] Lustre: server umount lustre-OST0000 complete [ 549.353189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 549.366771] Lustre: Skipped 1 previous similar message [ 559.587246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 559.592023] Lustre: Skipped 6 previous similar messages [ 562.143147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 562.309521] Lustre: server umount lustre-MDT0000 complete [ 564.705770] LustreError: 15093:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 564.732821] LustreError: 15093:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 566.492555] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684274 with bad export cookie 12419740506181695784 [ 566.501262] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 566.503898] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 566.824618] Lustre: server umount lustre-MDT0001 complete [ 572.605296] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:44:38 (1788684278) [ 579.676747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.069468] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 580.146882] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 585.192348] LustreError: 19322:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 585.228882] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 590.310217] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 593.918876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 598.921696] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 605.417493] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 612.177144] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 619.624988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 627.769704] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 636.840827] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 643.665944] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 650.720830] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 650.733884] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 650.754548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 656.814290] Lustre: server umount lustre-MDT0000 complete [ 658.919879] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 658.965045] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 660.882739] LustreError: 19302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684368 with bad export cookie 12419740506181696673 [ 660.893930] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 660.894697] LustreError: 19302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 661.358718] Lustre: server umount lustre-MDT0001 complete [ 676.274484] Lustre: server umount lustre-OST0000 complete [ 698.455364] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 709.382432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 709.862105] LustreError: 23703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 709.967205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 709.976499] Lustre: Skipped 2 previous similar messages [ 714.892234] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 724.648858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 730.693796] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 734.190672] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 741.713863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 748.694870] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 758.196774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 758.351926] Lustre: lustre-OST0001: new disk, initializing [ 758.355105] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 758.360092] Lustre: 26023:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 760.380663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 760.399936] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 760.566260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 764.974761] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 775.816263] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 779.287548] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 791.524328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 791.524408] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 791.544413] Lustre: Skipped 2 previous similar messages [ 791.552415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 791.567595] Lustre: Skipped 6 previous similar messages [ 797.680239] Lustre: server umount lustre-MDT0000 complete [ 801.763023] LustreError: 23704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 801.800223] LustreError: 23704:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 802.022195] LustreError: 23685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684509 with bad export cookie 12419740506181697520 [ 802.036983] LustreError: 23685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 802.037500] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 802.776577] Lustre: server umount lustre-MDT0001 complete [ 818.073771] Lustre: server umount lustre-OST0000 complete [ 832.987859] Lustre: server umount lustre-OST0001 complete [ 853.405979] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 864.586812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 864.958610] LustreError: 29041:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 865.041700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 865.047380] Lustre: Skipped 3 previous similar messages [ 868.641352] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 877.146615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 883.918298] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 888.125468] Lustre: 30189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 897.352359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 908.158144] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 918.586802] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 923.815680] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 931.938493] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 936.738040] Lustre: 32071:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 944.096819] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 944.107209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 944.123692] Lustre: Skipped 3 previous similar messages [ 944.126095] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 944.129944] Lustre: Skipped 4 previous similar messages [ 949.197654] Lustre: server umount lustre-MDT0000 complete [ 950.254564] LustreError: 31683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 950.274701] LustreError: 31683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 953.594189] LustreError: 29792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684661 with bad export cookie 12419740506181698815 [ 953.598472] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 953.615701] LustreError: 29792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 953.994608] Lustre: server umount lustre-MDT0001 complete [ 967.703908] Lustre: server umount lustre-OST0000 complete [ 970.733858] Lustre: 3650:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788684662/real 1788684662] req@ffff88ef47778000 x1875570934977664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788684678 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 972.342410] Lustre: server umount lustre-OST0001 complete [ 982.001892] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:51:28 (1788684688) [ 1011.778606] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1021.809045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1026.718818] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1035.735359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1041.149678] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1044.376087] Lustre: 36021:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1052.402257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1061.369744] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1071.186839] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1078.614707] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1088.407175] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1092.425684] Lustre: 37903:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1102.313821] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1102.333067] Lustre: Skipped 7 previous similar messages [ 1104.213068] Lustre: server umount lustre-MDT0000 complete [ 1107.434070] LustreError: 34876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1107.480206] LustreError: 34876:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 1109.275905] LustreError: 34855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684816 with bad export cookie 12419740506181699921 [ 1109.294234] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1109.668924] Lustre: server umount lustre-MDT0001 complete [ 1123.881719] Lustre: server umount lustre-OST0000 complete [ 1138.033946] Lustre: server umount lustre-OST0001 complete [ 1147.667601] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:54:13 (1788684853) [ 1164.245628] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1175.226554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1175.729880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1175.732772] Lustre: Skipped 7 previous similar messages [ 1181.012255] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1190.441463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1195.681525] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1198.815987] Lustre: 41193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1206.235750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1213.386724] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1222.928304] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1228.539801] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1236.965714] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1240.952165] Lustre: 43072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1241.801208] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1241.806303] Lustre: Skipped 1 previous similar message [ 1246.260124] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.18@tcp (stopping) [ 1246.265472] Lustre: Skipped 8 previous similar messages [ 1247.203685] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1247.221557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1251.665506] Lustre: server umount lustre-MDT0000 complete [ 1255.998627] LustreError: 40024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788684963 with bad export cookie 12419740506181701020 [ 1256.003070] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1256.014385] LustreError: 40024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1262.387545] Lustre: server umount lustre-MDT0001 complete [ 1269.797721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1281.536728] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xac5bc830117d920a [ 1281.558188] Lustre: MGC192.168.201.118@tcp: Connection restored to 0@lo (at 0@lo) [ 1287.494854] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1291.442106] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1298.548685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1304.571508] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1304.589880] Lustre: Skipped 1 previous similar message [ 1306.407201] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1315.737608] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1317.817469] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1324.511262] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1326.615547] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1334.777265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1339.930239] Lustre: server umount lustre-MDT0000 complete [ 1344.085761] LustreError: 40024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788685051 with bad export cookie 12419740506181702154 [ 1344.100303] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1344.639194] Lustre: server umount lustre-MDT0001 complete [ 1359.444512] Lustre: server umount lustre-OST0000 complete [ 1361.890039] Lustre: 3649:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788685053/real 1788685053] req@ffff88f045a9d180 x1875570935120256/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788685069 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1364.249290] Lustre: server umount lustre-OST0001 complete [ 1374.171931] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:58:00 (1788685080) [ 1393.728061] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1405.327028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1405.631866] LustreError: 47687:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1405.651877] LustreError: 47687:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 34 previous similar messages [ 1410.565377] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1420.033486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1425.794952] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1429.746257] Lustre: 48841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1438.582138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1447.612862] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1459.074625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1468.114496] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1477.503958] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1482.356587] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1497.066182] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1497.077195] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1497.098070] Lustre: Skipped 10 previous similar messages [ 1502.919425] Lustre: server umount lustre-MDT0000 complete [ 1507.320533] LustreError: 47668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788685214 with bad export cookie 12419740506181702938 [ 1507.330481] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1507.338988] LustreError: 47668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1507.835153] Lustre: server umount lustre-MDT0001 complete [ 1521.905952] Lustre: server umount lustre-OST0000 complete [ 1537.084868] Lustre: server umount lustre-OST0001 complete [ 1548.574580] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 05:00:54 (1788685254) [ 1552.539948] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1555.682567] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 05:01:00 (1788685260) [ 1569.841251] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 1580.772520] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1605.968702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1606.365890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1606.487255] Lustre: lustre-MDT0000: new disk, initializing [ 1606.677471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1611.721756] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1622.081782] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1629.682086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1630.020750] Lustre: lustre-OST0000: new disk, initializing [ 1630.029678] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1630.037883] Lustre: Skipped 1 previous similar message [ 1630.048682] Lustre: 54934:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1631.205052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1631.217483] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1631.250135] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1637.233774] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1647.067584] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1649.329405] Lustre: DEBUG MARKER: create test files [ 1687.520413] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1692.486457] Lustre: server umount lustre-OST0000 complete [ 1697.254491] Lustre: server umount lustre-MDT0000 complete [ 1711.096138] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1714.674160] Key type lgssc unregistered [ 1715.072392] LNet: 56565:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1715.100207] LNetError: 56565:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1715.123670] LNet: Removed LNI 192.168.201.118@tcp [ 1716.403080] Key type .llcrypt unregistered [ 1716.404801] Key type ._llcrypt unregistered [ 1745.653674] Key type ._llcrypt registered [ 1745.657916] Key type .llcrypt registered [ 1745.766800] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1747.414126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1747.440173] alg: No test for adler32 (adler32-zlib) [ 1748.799939] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 1749.237596] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1751.129588] Key type lgssc registered [ 1752.131476] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1761.743873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1761.759287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1763.078491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1766.585465] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1775.234398] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1782.850671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1783.167438] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1788.282330] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1788.382512] Lustre: 59294:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1788.390946] Lustre: 59294:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1788.396042] Lustre: 59294:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1788.402511] Lustre: 59294:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1788.410851] Lustre: 59294:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1788.414714] Lustre: 59294:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1790.233201] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1800.287903] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1805.886266] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1812.822433] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1819.937214] Lustre: server umount lustre-OST0000 complete [ 1834.657672] Lustre: server umount lustre-MDT0000 complete [ 1845.685617] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1849.256623] Key type lgssc unregistered [ 1849.681780] LNet: 60906:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1849.689333] LNetError: 60906:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1849.705861] LNet: Removed LNI 192.168.201.118@tcp [ 1850.851311] Key type .llcrypt unregistered [ 1850.854750] Key type ._llcrypt unregistered [ 1882.211825] Key type ._llcrypt registered [ 1882.216404] Key type .llcrypt registered [ 1882.328594] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1884.105619] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1884.174931] alg: No test for adler32 (adler32-zlib) [ 1885.296453] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 1885.622286] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1887.417080] Key type lgssc registered [ 1889.011416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1898.364910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1898.391403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1899.968700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1905.104855] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1913.334447] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1920.570846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1920.961462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1925.055376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1925.179343] Lustre: 63737:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1925.185507] Lustre: 63737:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1925.193674] Lustre: 63737:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1925.200638] Lustre: 63737:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1925.204711] Lustre: 63737:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1925.213964] Lustre: 63737:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1927.539790] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1936.892528] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1942.969834] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1981.408874] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1981.422151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1986.528645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1986.947803] Lustre: server umount lustre-OST0000 complete [ 1990.253308] Lustre: server umount lustre-MDT0000 complete [ 2000.797280] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 2004.132334] Key type lgssc unregistered [ 2004.495138] LNet: 65321:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2004.509028] LNetError: 65321:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2004.529656] LNet: Removed LNI 192.168.201.118@tcp [ 2005.586621] Key type .llcrypt unregistered [ 2005.593318] Key type ._llcrypt unregistered [ 2024.556558] Key type ._llcrypt registered [ 2024.559432] Key type .llcrypt registered [ 2024.669390] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 2042.346950] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2043.429465] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2043.555230] alg: No test for adler32 (adler32-zlib) [ 2044.647382] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 2044.892378] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 2046.615202] Key type lgssc registered [ 2047.925746] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2086.001491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2086.028356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2087.269253] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2087.291164] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2087.348316] Lustre: lustre-MDT0000: new disk, initializing [ 2087.429309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2087.455305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2091.975715] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2103.536603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2103.650992] Lustre: 68472:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2103.700528] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2103.704736] Lustre: Skipped 1 previous similar message [ 2103.763611] Lustre: lustre-MDT0001: new disk, initializing [ 2103.848809] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2103.901676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2103.910141] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2108.751481] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2118.336553] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2124.438471] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2131.105231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2131.400345] Lustre: lustre-OST0000: new disk, initializing [ 2131.406662] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2131.415502] Lustre: 70442:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2131.542310] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2137.172238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2137.191905] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2137.379175] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2138.165422] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2146.867460] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2153.081358] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2153.381834] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2159.576060] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2159.921682] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2163.172963] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2163.174768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2163.204937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2163.219667] Lustre: Skipped 1 previous similar message [ 2168.291528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2168.307781] Lustre: Skipped 2 previous similar messages [ 2169.163438] Lustre: server umount lustre-OST0000 complete [ 2173.411995] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2173.432586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2173.436398] Lustre: Skipped 1 previous similar message [ 2178.533115] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2178.544096] Lustre: Skipped 2 previous similar messages [ 2180.059108] Lustre: server umount lustre-MDT0000 complete [ 2184.809926] LustreError: 68466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788685892 with bad export cookie 16952647914377620027 [ 2184.814577] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2184.819551] LustreError: 68466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2185.439617] Lustre: server umount lustre-MDT0001 complete [ 2194.427155] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:11:40 (1788685900) [ 2207.836678] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2250.933377] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 2266.498483] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2308.099321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2308.339260] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2308.386232] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2308.490731] Lustre: lustre-MDT0000: new disk, initializing [ 2308.612177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2308.637076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2313.597579] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2324.768623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2324.956365] Lustre: 76474:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2325.020623] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2325.025992] Lustre: Skipped 1 previous similar message [ 2325.115991] Lustre: lustre-MDT0001: new disk, initializing [ 2325.258859] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2325.324538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2325.334404] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2330.623568] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2341.673497] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2349.236104] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2357.058714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2357.405190] Lustre: lustre-OST0000: new disk, initializing [ 2357.412936] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2357.419496] Lustre: 78445:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2357.571009] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2363.747056] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2366.056303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2366.070684] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2366.225352] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2374.884285] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2380.104275] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2380.334537] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2386.248616] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2386.672580] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2392.033811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2392.046076] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2392.059385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2395.691568] Lustre: server umount lustre-OST0000 complete [ 2402.272259] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.284392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2402.294563] Lustre: Skipped 2 previous similar messages [ 2402.297270] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2402.307426] Lustre: Skipped 2 previous similar messages [ 2402.334497] LustreError: Skipped 1 previous similar message [ 2406.902812] Lustre: server umount lustre-MDT0000 complete [ 2407.395659] LustreError: 77653:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.425383] LustreError: 77653:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2411.259288] LustreError: 76466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788686118 with bad export cookie 16952647914377620909 [ 2411.262561] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2411.274556] LustreError: 76466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2411.828516] Lustre: server umount lustre-MDT0001 complete [ 2423.333705] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:15:29 (1788686129) [ 2431.997068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2432.334762] LustreError: 80860:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2432.403786] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2437.601803] LustreError: 80861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2437.788825] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2441.706747] LustreError: 80860:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2446.818762] LustreError: 80861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2447.218054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2453.418804] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2462.447815] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2467.533934] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2475.027366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2475.249269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2475.256954] Lustre: Skipped 1 previous similar message [ 2481.167311] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2489.203781] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2494.149052] Lustre: 83532:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 18e04f98-fe54-4d55-8a2e-4d4e19dbe248 at adminstrative request [ 2498.049692] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2498.071920] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2508.260475] Lustre: MGS: Client 18e04f98-fe54-4d55-8a2e-4d4e19dbe248 (at 0@lo) reconnecting [ 2508.273733] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xeb43e7758abeb11f to 0xeb43e7758abeb3bf [ 2508.287288] Lustre: MGC192.168.201.118@tcp: Connection restored to 0@lo (at 0@lo) [ 2532.307278] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2532.540710] Lustre: lustre-OST0001: new disk, initializing [ 2532.543233] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2532.548175] Lustre: 83926:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2539.106869] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2541.695606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2541.712683] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2541.974707] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2549.947596] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2557.419944] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2557.439197] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2557.466724] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2557.476578] Lustre: Skipped 1 previous similar message [ 2559.661755] Lustre: server umount lustre-OST0001 complete [ 2562.538757] LustreError: 84395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2562.566501] LustreError: 84395:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2564.590458] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2564.596980] LustreError: Skipped 1 previous similar message [ 2569.381345] Lustre: server umount lustre-OST0000 complete [ 2575.842924] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2575.846713] LustreError: Skipped 1 previous similar message [ 2575.862209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2575.889843] Lustre: Skipped 3 previous similar messages [ 2579.545683] Lustre: server umount lustre-MDT0000 complete [ 2582.454877] LustreError: 80840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788686289 with bad export cookie 16952647914377622463 [ 2582.461069] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2582.466808] LustreError: 80840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2582.723080] Lustre: server umount lustre-MDT0001 complete [ 2590.928892] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:18:17 (1788686297) [ 2592.475617] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2594.404817] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:18:20 (1788686300) [ 2596.142918] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2598.168046] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:18:24 (1788686304) [ 2599.979320] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2601.638233] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:18:28 (1788686308) [ 2603.344825] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2605.022578] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:18:31 (1788686311) [ 2607.005691] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2614.287442] loop: module loaded [ 2614.309494] loop0: detected capacity change from 0 to 4294967296 [ 2614.425272] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2616.315654] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2620.943387] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2626.146697] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2653.904136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2653.932264] LustreError: 88053:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2653.943138] LustreError: 88053:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2653.950431] LustreError: 88053:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2653.957924] LustreError: 88053:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2672.829186] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 2684.057730] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2720.494297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2720.650493] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2720.682899] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2720.766724] Lustre: lustre-MDT0000: new disk, initializing [ 2720.891401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2720.895864] Lustre: Skipped 1 previous similar message [ 2720.909346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2725.881932] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2737.089432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2737.261262] Lustre: 90918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2737.522725] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2743.376778] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2753.042909] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2757.985837] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2764.267709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2764.630551] Lustre: 92886:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2771.888896] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2774.085018] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2774.181198] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2781.790962] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2786.383757] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2786.595677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2790.689141] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2790.957408] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2794.979535] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2794.979559] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2794.979566] Lustre: Skipped 1 previous similar message [ 2794.989713] LustreError: Skipped 1 previous similar message [ 2795.011800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2795.020170] Lustre: Skipped 8 previous similar messages [ 2798.845779] Lustre: server umount lustre-OST0000 complete [ 2809.392321] Lustre: server umount lustre-MDT0000 complete [ 2810.338293] LustreError: 91931:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2810.364783] LustreError: 91931:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2814.134348] LustreError: 92885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788686521 with bad export cookie 16952647914377623079 [ 2814.138947] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2814.148728] LustreError: 92885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2814.797745] Lustre: server umount lustre-MDT0001 complete [ 2823.891368] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:22:10 (1788686530) [ 2830.985253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2836.016599] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2845.672691] LustreError: 95298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2845.696869] LustreError: 95298:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2845.997948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2851.529558] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2860.065785] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2866.250250] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2872.834081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2873.152369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2873.155991] Lustre: Skipped 4 previous similar messages [ 2880.767642] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2888.220960] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2892.993297] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2893.439461] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2901.088111] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2901.291089] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2908.929153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2909.208349] Lustre: lustre-OST0001: new disk, initializing [ 2909.213721] Lustre: Skipped 2 previous similar messages [ 2909.221641] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2909.228660] Lustre: Skipped 3 previous similar messages [ 2909.250670] Lustre: 98671:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2915.300059] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2917.670756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2917.697057] Lustre: Skipped 2 previous similar messages [ 2917.726755] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2917.914761] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2929.424566] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2935.037474] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2935.375139] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2940.498492] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2940.836744] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2960.776552] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2986.336647] Lustre: Failing over lustre-MDT0000 [ 2988.852593] Lustre: server umount lustre-MDT0000 complete [ 2989.137215] LustreError: 97140:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2989.150201] LustreError: 97140:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2989.544209] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2989.563639] Lustre: Skipped 4 previous similar messages [ 3005.931851] Lustre: 66696:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788686697/real 1788686697] req@00000000fd903bd8 x1875572951366656/t0(0) o400->MGC192.168.201.118@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788686713 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3005.960276] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3014.935688] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3014.938312] LDISKFS-fs (dm-0): recovery complete [ 3014.946830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3016.163546] LustreError: 101488:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3016.176772] LustreError: 101488:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@0000000001e40ffb x1875572951372416/t0(0) o250->MGC192.168.201.118@tcp@0@lo:26/25 lens 520/544 e 0 to 1 dl 1788686723 ref 1 fl Rpc:EXNQU/200/ffffffff rc -110/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3016.240478] LustreError: 101488:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3016.262834] LustreError: 66693:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000af03abcb x1875572951376128/t0(0) o250->MGC192.168.201.118@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3016.292439] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xeb43e7758abf7e3a [ 3016.308377] Lustre: MGC192.168.201.118@tcp: Connection restored to 0@lo (at 0@lo) [ 3016.786817] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3020.837324] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3020.838021] Lustre: 95293:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3021.756697] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3021.796976] Lustre: 96026:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3021.820111] Lustre: 96026:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 3021.834039] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3021.950695] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3022.281296] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3022.615543] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3022.624648] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3022.958164] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3022.978878] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3023.004990] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3023.033547] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 3023.720172] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3023.730501] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3024.072119] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3024.079438] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3025.034775] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3025.041570] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3025.990147] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3026.002941] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3026.337890] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3026.346332] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3029.199468] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3029.219723] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 3030.184456] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3030.208685] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3030.550755] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3030.564826] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3037.218435] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3037.229788] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 3038.193753] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3038.219507] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3038.855224] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3038.860735] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 3053.443586] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3053.455773] Lustre: 101519:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 3054.401235] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3054.415971] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3055.063267] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3055.076030] LustreError: 101519:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3080.023140] Lustre: 101519:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3080.039616] Lustre: 101519:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3080.052841] Lustre: 101519:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1a8779f5-ef89-477f-987d-ae9d6c5da30b@192.168.201.18@tcp [ 3080.093890] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3080.102126] Lustre: 101519:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000005bf361c7 x1875572933334912/t0(8589934779) o101->1a8779f5-ef89-477f-987d-ae9d6c5da30b@192.168.201.18@tcp:443/0 lens 592/0 e 0 to 0 dl 1788686798 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3080.141339] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3080.157404] Lustre: lustre-MDT0000: Denying connection for new client 1a8779f5-ef89-477f-987d-ae9d6c5da30b (at 192.168.201.18@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3080.187991] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3080.241429] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3080.249048] Lustre: Skipped 2 previous similar messages [ 3080.255807] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3080.296738] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3080.297495] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3095.526719] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3095.535574] LustreError: Skipped 1 previous similar message [ 3095.538417] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3095.545229] Lustre: Skipped 1 previous similar message [ 3095.560526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3095.568660] Lustre: Skipped 3 previous similar messages [ 3101.838723] Lustre: server umount lustre-OST0000 complete [ 3111.741873] Lustre: server umount lustre-OST0001 complete [ 3121.835688] Lustre: server umount lustre-MDT0000 complete [ 3124.193690] LustreError: 98494:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3124.208463] LustreError: 98494:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 45 previous similar messages [ 3125.686147] LustreError: 95278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788686833 with bad export cookie 16952647914377674298 [ 3125.693033] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3125.693330] LustreError: 95278:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3126.050829] Lustre: server umount lustre-MDT0001 complete [ 3133.437880] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:27:20 (1788686840) [ 3139.936356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3139.943494] Lustre: *** cfs_fail_loc=197, val=0*** [ 3155.423728] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3160.544395] LustreError: 103494:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.118@tcp: failed processing log, type 4: rc = -110 [ 3165.689540] Lustre: *** cfs_fail_loc=197, val=0*** [ 3165.692177] Lustre: Skipped 65 previous similar messages [ 3186.271342] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3186.276801] Lustre: Skipped 2 previous similar messages [ 3192.365590] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3200.065452] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3202.370584] Lustre: server umount lustre-OST0000 complete [ 3210.491242] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:28:37 (1788686917) [ 3231.935264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3237.334938] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3246.451903] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3250.311107] LustreError: 105146:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3250.320719] LustreError: 105146:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3250.335992] LustreError: 105146:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3250.667439] Lustre: server umount lustre-MDT0000 complete [ 3285.175442] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3295.345779] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3334.499849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3334.719924] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3334.748188] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3334.844312] Lustre: lustre-MDT0000: new disk, initializing [ 3334.920590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3340.659928] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3352.368317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3352.491290] Lustre: 109328:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3352.856023] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3357.754794] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3367.911323] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3373.916686] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3381.810004] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3382.350943] Lustre: 111298:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3389.576458] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3391.371536] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3400.903851] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3405.999679] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3406.283338] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3410.807210] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3411.040875] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3417.058650] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3417.082106] Lustre: Skipped 5 previous similar messages [ 3417.082969] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3417.107613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3417.107669] LustreError: Skipped 4 previous similar messages [ 3417.129700] Lustre: Skipped 11 previous similar messages [ 3427.299397] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3427.481765] Lustre: server umount lustre-OST0000 complete [ 3437.543492] LustreError: 110391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3437.557603] LustreError: 110391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3437.849796] Lustre: server umount lustre-MDT0000 complete [ 3441.878220] LustreError: 109321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788687149 with bad export cookie 16952647914377684966 [ 3441.881467] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3441.889650] LustreError: 109321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3442.381582] Lustre: server umount lustre-MDT0001 complete [ 3450.864397] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:32:37 (1788687157) [ 3459.590449] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3462.175858] Key type lgssc unregistered [ 3462.468214] LNet: 113785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3462.475745] LNetError: 113785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3462.495249] LNet: Removed LNI 192.168.201.118@tcp [ 3463.420654] Key type .llcrypt unregistered [ 3463.422413] Key type ._llcrypt unregistered [ 3481.999427] Key type ._llcrypt registered [ 3482.001506] Key type .llcrypt registered [ 3482.091637] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3496.766379] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3497.751755] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3497.777150] alg: No test for adler32 (adler32-zlib) [ 3498.941980] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 3499.155858] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3500.847187] Key type lgssc registered [ 3501.859906] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3546.179980] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3546.210438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3547.623094] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3547.672559] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3547.792173] Lustre: lustre-MDT0000: new disk, initializing [ 3547.960773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3548.036424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3553.408455] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3565.562928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3565.732147] Lustre: lustre-OST0000: new disk, initializing [ 3565.737473] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3565.740149] Lustre: Skipped 1 previous similar message [ 3565.743329] Lustre: 117967:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3565.806379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3572.989131] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3573.004626] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3573.083679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3573.125574] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3584.141811] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3584.388820] Lustre: lustre-OST0001: new disk, initializing [ 3584.395645] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3584.406802] Lustre: 118830:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3584.528162] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3591.691306] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3592.317487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3592.327534] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3592.392330] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3602.404814] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3602.422202] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3606.820648] Lustre: server umount lustre-MDT0000 complete [ 3613.254917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3617.811076] LustreError: 117153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788687325 with bad export cookie 6908621424356423295 [ 3617.814059] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3617.818713] LustreError: 117153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3618.023928] Lustre: server umount lustre-OST0000 complete [ 3621.894552] Lustre: server umount lustre-OST0001 complete [ 3630.186611] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:35:36 (1788687336) [ 3653.657977] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:36:00 (1788687360) [ 3664.400232] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3687.058343] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:36:33 (1788687393) [ 3727.338632] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3738.141702] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3778.630654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3778.922741] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3778.979556] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3779.103511] Lustre: lustre-MDT0000: new disk, initializing [ 3779.245333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3779.273334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3784.010578] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3795.561977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3795.747363] Lustre: 126582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3795.856904] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3795.861963] Lustre: Skipped 1 previous similar message [ 3795.942156] Lustre: lustre-MDT0001: new disk, initializing [ 3796.028358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3796.070061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3796.082577] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3800.637632] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3810.018646] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3816.907822] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3824.422685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3824.736482] Lustre: lustre-OST0000: new disk, initializing [ 3824.741478] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3824.747490] Lustre: 128556:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3824.862527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3830.330951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3830.347258] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3830.507521] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3831.662636] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3841.387960] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3846.852977] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3847.064965] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3852.778768] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3853.105077] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3856.354327] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3856.354624] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3856.359654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3856.359662] Lustre: Skipped 1 previous similar message [ 3856.372541] LustreError: Skipped 1 previous similar message [ 3856.430230] Lustre: Skipped 2 previous similar messages [ 3861.486747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3861.494408] Lustre: Skipped 2 previous similar messages [ 3861.974614] Lustre: server umount lustre-OST0000 complete [ 3866.595160] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3866.602765] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3866.623688] Lustre: Skipped 1 previous similar message [ 3866.642281] Lustre: Skipped 1 previous similar message [ 3871.720194] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3871.725981] Lustre: Skipped 1 previous similar message [ 3872.128793] Lustre: server umount lustre-MDT0000 complete [ 3875.690743] LustreError: 128555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788687583 with bad export cookie 6908621424356424520 [ 3875.707922] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3876.006655] Lustre: server umount lustre-MDT0001 complete [ 3885.063780] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:39:51 (1788687591) [ 3891.573865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3892.075615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3896.428270] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3897.316631] LustreError: 130969:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3897.338593] LustreError: 130969:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3902.434266] LustreError: 130968:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3904.435475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3907.952444] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3916.880304] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3923.017665] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3930.777865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3931.224216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3931.238473] Lustre: Skipped 1 previous similar message [ 3938.235046] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3946.491376] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3954.052503] Lustre: *** cfs_fail_loc=159, val=0*** [ 3970.718924] Lustre: lustre-MDT0000: Client 8ada824b-5b28-449c-ab5d-d37e442ad9c5 (at 192.168.201.18@tcp) reconnecting [ 3972.114806] Lustre: *** cfs_fail_loc=159, val=0*** [ 3972.125285] Lustre: Skipped 5 previous similar messages [ 3988.624748] Lustre: lustre-MDT0000: Client 8ada824b-5b28-449c-ab5d-d37e442ad9c5 (at 192.168.201.18@tcp) reconnecting [ 4006.376871] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4006.380575] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4006.397645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4006.428594] Lustre: Skipped 1 previous similar message [ 4011.514708] LustreError: 133459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4011.619446] Lustre: server umount lustre-OST0000 complete [ 4017.632466] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4017.641962] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4017.661130] Lustre: Skipped 1 previous similar message [ 4023.327689] Lustre: server umount lustre-MDT0000 complete [ 4026.856610] LustreError: 130968:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4026.869428] LustreError: 130968:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4026.983433] LustreError: 130949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788687734 with bad export cookie 6908621424356425395 [ 4026.986189] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4026.992383] LustreError: 130949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 4027.648075] Lustre: server umount lustre-MDT0001 complete [ 4040.857458] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4044.509479] Key type lgssc unregistered [ 4044.877151] LNet: 134751:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4044.888530] LNetError: 134751:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4044.904740] LNet: Removed LNI 192.168.201.118@tcp [ 4046.067482] Key type .llcrypt unregistered [ 4046.069653] Key type ._llcrypt unregistered [ 4073.441773] Key type ._llcrypt registered [ 4073.444356] Key type .llcrypt registered [ 4073.602305] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4075.036250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4075.072739] alg: No test for adler32 (adler32-zlib) [ 4076.177160] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 4076.424794] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4078.119221] Key type lgssc registered [ 4079.623504] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4087.797991] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4087.808407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4089.390109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4094.747658] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4098.534926] LustreError: 136161:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4098.556212] LustreError: 136161:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4103.651596] LustreError: 136162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4104.068454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4104.506965] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4109.311570] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4117.722424] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4124.699815] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4131.584648] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4131.884061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4133.963703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4139.147913] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4148.286869] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4158.421947] Lustre: *** cfs_fail_loc=159, val=0*** [ 4174.921838] Lustre: lustre-MDT0000: Client 333f38d6-33e3-4ee8-877f-3c151119993e (at 192.168.201.18@tcp) reconnecting [ 4177.544237] Lustre: *** cfs_fail_loc=159, val=0*** [ 4193.958191] Lustre: lustre-MDT0001: Client 333f38d6-33e3-4ee8-877f-3c151119993e (at 192.168.201.18@tcp) reconnecting [ 4195.599045] Lustre: *** cfs_fail_loc=159, val=0*** [ 4195.611113] Lustre: Skipped 3 previous similar messages [ 4211.364290] Lustre: lustre-MDT0001: Client 333f38d6-33e3-4ee8-877f-3c151119993e (at 192.168.201.18@tcp) reconnecting [ 4215.351501] Lustre: 139200:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4218.403156] Lustre: *** cfs_fail_loc=159, val=0*** [ 4218.412352] Lustre: Skipped 4 previous similar messages [ 4234.892627] Lustre: lustre-MDT0000: Client 5585a296-ae05-4f2c-8043-d24cc3cc749c (at 192.168.201.18@tcp) reconnecting [ 4237.033405] Lustre: *** cfs_fail_loc=159, val=0*** [ 4237.035180] Lustre: Skipped 14 previous similar messages [ 4253.281275] Lustre: lustre-MDT0000: Client 5585a296-ae05-4f2c-8043-d24cc3cc749c (at 192.168.201.18@tcp) reconnecting [ 4254.961514] Lustre: 139492:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4277.216951] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4277.228906] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4277.259740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4277.731767] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4277.754722] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4277.781525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4281.148703] Lustre: server umount lustre-OST0000 complete [ 4287.973152] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4288.002864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4288.013033] Lustre: Skipped 1 previous similar message [ 4292.679350] Lustre: server umount lustre-MDT0000 complete [ 4293.088704] LustreError: 136161:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4293.096759] LustreError: 136161:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4297.070600] LustreError: 138004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788688004 with bad export cookie 16919011192158864921 [ 4297.075777] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4297.083593] LustreError: 138004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4297.496961] Lustre: server umount lustre-MDT0001 complete [ 4312.116712] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4315.757698] Key type lgssc unregistered [ 4316.071775] LNet: 140572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4316.077557] LNetError: 140572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4316.093738] LNet: Removed LNI 192.168.201.118@tcp [ 4317.105266] Key type .llcrypt unregistered [ 4317.107817] Key type ._llcrypt unregistered [ 4347.358356] Key type ._llcrypt registered [ 4347.360506] Key type .llcrypt registered [ 4347.499505] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4348.963997] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4349.063507] alg: No test for adler32 (adler32-zlib) [ 4350.097246] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 4350.593205] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4352.343257] Key type lgssc registered [ 4354.032898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4364.197404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4364.216379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4365.882897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4371.147267] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4376.036679] LustreError: 141988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4376.063506] LustreError: 141988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4381.157890] LustreError: 141989:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4381.851772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4382.321566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4388.051255] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4396.264119] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4403.019127] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4411.281787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4411.688830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4418.792695] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4427.084596] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4427.106615] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4436.279533] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4444.369755] Lustre: 144590:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4449.071895] Lustre: 144639:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4450.796870] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4451.968851] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4456.651798] Lustre: 144785:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4457.897858] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4458.790687] Lustre: 144883:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4467.176615] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4467.177309] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4467.187675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4467.206405] Lustre: Skipped 1 previous similar message [ 4469.574956] Lustre: server umount lustre-OST0000 complete [ 4474.336158] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4474.339376] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4474.380181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4474.386094] Lustre: Skipped 1 previous similar message [ 4477.408961] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4477.415891] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4477.464372] Lustre: Skipped 1 previous similar message [ 4480.063243] Lustre: server umount lustre-MDT0000 complete [ 4482.528559] LustreError: 141984:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4482.540648] LustreError: 141984:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4484.828689] LustreError: 141970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788688192 with bad export cookie 10680164205511235535 [ 4484.838213] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4484.849283] LustreError: 141970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4485.372898] Lustre: server umount lustre-MDT0001 complete [ 4496.521494] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4499.591424] Key type lgssc unregistered [ 4500.092663] LNet: 145946:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4500.097452] LNetError: 145946:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4500.107241] LNet: Removed LNI 192.168.201.118@tcp [ 4501.004909] Key type .llcrypt unregistered [ 4501.009989] Key type ._llcrypt unregistered [ 4533.279369] Key type ._llcrypt registered [ 4533.281467] Key type .llcrypt registered [ 4533.479911] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4534.790248] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4534.912767] alg: No test for adler32 (adler32-zlib) [ 4536.399341] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 4536.734737] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4538.463705] Key type lgssc registered [ 4540.002585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4549.038695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4549.051912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.630914] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4556.113866] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4560.864357] LustreError: 147360:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4560.886374] LustreError: 147360:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4565.596157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4566.014783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4571.270424] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4579.314330] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4585.266449] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4591.874436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4592.214334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4596.410146] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4599.850806] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4608.610535] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4610.747894] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4615.723926] Lustre: *** cfs_fail_loc=159, val=0*** [ 4615.736351] Lustre: Skipped 2 previous similar messages [ 4631.127237] Lustre: lustre-MDT0000: Client 53f98f29-bbed-4cd1-bc9e-8e20f5b61020 (at 192.168.201.18@tcp) reconnecting [ 4641.761642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4641.764619] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4641.785595] Lustre: Skipped 1 previous similar message [ 4641.821946] Lustre: Skipped 1 previous similar message [ 4643.906363] Lustre: server umount lustre-OST0000 complete [ 4647.904851] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4647.911562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4647.923323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4652.003739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4652.017215] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4654.301121] Lustre: server umount lustre-MDT0000 complete [ 4657.121241] LustreError: 147361:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4657.158054] LustreError: 147361:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4658.125443] LustreError: 147340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788688365 with bad export cookie 5870885819563119564 [ 4658.136600] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4658.149944] LustreError: 147340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4658.529947] Lustre: server umount lustre-MDT0001 complete [ 4672.006630] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4675.561808] Key type lgssc unregistered [ 4675.859513] LNet: 151022:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4675.869825] LNetError: 151022:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4675.890481] LNet: Removed LNI 192.168.201.118@tcp [ 4676.997813] Key type .llcrypt unregistered [ 4677.001523] Key type ._llcrypt unregistered [ 4705.225711] Key type ._llcrypt registered [ 4705.228300] Key type .llcrypt registered [ 4705.381089] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4707.079955] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4707.119077] alg: No test for adler32 (adler32-zlib) [ 4708.305597] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 4708.669588] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4710.319211] Key type lgssc registered [ 4711.259996] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4720.120297] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4720.186402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4721.749276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4727.500548] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4731.879066] LustreError: 152434:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4731.901474] LustreError: 152434:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4736.871872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4737.023899] LustreError: 152433:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4737.333936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4742.325480] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4750.988557] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4758.148584] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4767.332622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4767.813653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4776.026738] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4782.136929] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4783.627057] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4792.389866] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4814.879528] Lustre: lustre-MDT0001: evicting f29e7de6-14f1-4527-9785-a1783323c52e (at 192.168.201.18@tcp) by administrative request [ 4814.891358] Lustre: Skipped 1 previous similar message [ 4848.612238] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4848.615927] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4848.628525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4848.635798] Lustre: Skipped 1 previous similar message [ 4848.654142] LustreError: Skipped 1 previous similar message [ 4853.733086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4853.738417] Lustre: Skipped 1 previous similar message [ 4858.335302] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4858.551937] Lustre: server umount lustre-OST0000 complete [ 4863.969992] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4863.971727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4863.987509] Lustre: Skipped 1 previous similar message [ 4864.008020] Lustre: Skipped 1 previous similar message [ 4868.905073] Lustre: server umount lustre-MDT0000 complete [ 4869.096574] LustreError: 153163:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4869.111844] LustreError: 153163:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4873.488465] LustreError: 152413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788688581 with bad export cookie 14473385076222519333 [ 4873.499078] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4874.119591] Lustre: server umount lustre-MDT0001 complete [ 4888.655277] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4891.469503] Key type lgssc unregistered [ 4891.791668] LNet: 156528:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4891.802109] LNetError: 156528:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4891.827637] LNet: Removed LNI 192.168.201.118@tcp [ 4892.813061] Key type .llcrypt unregistered [ 4892.815839] Key type ._llcrypt unregistered [ 4974.095429] Key type ._llcrypt registered [ 4974.099130] Key type .llcrypt registered [ 4974.214788] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4975.764129] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4975.797554] alg: No test for adler32 (adler32-zlib) [ 4976.892183] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 4977.202598] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4978.890212] Key type lgssc registered [ 4980.183444] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4988.843769] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4988.869854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4990.466292] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4995.781026] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4999.664443] LustreError: 159020:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4999.696446] LustreError: 159020:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5004.174582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5004.624325] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5009.039107] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5015.800258] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5021.162702] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5027.195781] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5027.606975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5028.669194] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5034.587869] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5034.812493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5042.957412] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5054.434212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5054.437281] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5054.458397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5055.459928] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5055.462404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5055.472634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5055.500735] Lustre: Skipped 1 previous similar message [ 5060.586216] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5060.591725] Lustre: Skipped 1 previous similar message [ 5065.703322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5065.723036] Lustre: Skipped 1 previous similar message [ 5066.207163] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5066.509972] Lustre: server umount lustre-OST0000 complete [ 5070.821953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5070.845119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5070.854050] Lustre: Skipped 1 previous similar message [ 5077.046086] Lustre: server umount lustre-MDT0000 complete [ 5081.061963] LustreError: 159750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5081.087155] LustreError: 159750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5081.103202] LustreError: 161605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788688788 with bad export cookie 12326811351064722058 [ 5081.105838] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5081.118327] LustreError: 161605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5081.557370] Lustre: server umount lustre-MDT0001 complete [ 5093.232897] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 5096.343501] Key type lgssc unregistered [ 5096.600087] LNet: 162559:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5096.606787] LNetError: 162559:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5096.637321] LNet: Removed LNI 192.168.201.118@tcp [ 5097.857614] Key type .llcrypt unregistered [ 5097.864428] Key type ._llcrypt unregistered [ 5125.960563] Key type ._llcrypt registered [ 5125.962490] Key type .llcrypt registered [ 5126.048955] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5126.917199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5126.951124] alg: No test for adler32 (adler32-zlib) [ 5128.182682] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 5128.576327] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 5130.359183] Key type lgssc registered [ 5131.921434] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5145.901918] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 06:00:52 (1788688852) [ 5165.741801] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 5177.042979] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5215.312867] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5215.340278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5216.603642] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5216.662107] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5216.768703] Lustre: lustre-MDT0000: new disk, initializing [ 5216.895236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5216.913333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5222.297874] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5233.593168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5233.823683] Lustre: 166843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5233.898677] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5233.902963] Lustre: Skipped 1 previous similar message [ 5234.091614] Lustre: lustre-MDT0001: new disk, initializing [ 5234.186897] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5234.240777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5234.251173] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5238.784585] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5248.524325] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5254.686690] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5261.986626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5262.598854] Lustre: lustre-OST0000: new disk, initializing [ 5262.604250] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5262.612220] Lustre: 168810:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5262.749618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5270.124351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5270.144667] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5270.324695] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5270.592297] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5281.209771] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5290.976101] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5290.982418] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5290.990222] LustreError: Skipped 1 previous similar message [ 5290.994578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5291.005958] Lustre: Skipped 1 previous similar message [ 5296.102243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5296.116820] Lustre: Skipped 2 previous similar messages [ 5296.889570] Lustre: server umount lustre-OST0000 complete [ 5300.707124] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5300.713796] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5300.729756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5306.337523] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5306.348386] Lustre: Skipped 3 previous similar messages [ 5308.807850] Lustre: server umount lustre-MDT0000 complete [ 5312.199949] LustreError: 166835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788689019 with bad export cookie 7934127285334603474 [ 5312.205340] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5312.210825] LustreError: 166835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5312.450871] Lustre: server umount lustre-MDT0001 complete [ 5322.874892] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 5325.425561] Key type lgssc unregistered [ 5325.732884] LNet: 170718:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5325.744984] LNetError: 170718:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5326.832169] LNet: Removed LNI 192.168.201.118@tcp [ 5327.761271] Key type .llcrypt unregistered [ 5327.762979] Key type ._llcrypt unregistered [ 5353.639975] Key type ._llcrypt registered [ 5353.642792] Key type .llcrypt registered [ 5353.772199] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5355.340331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5355.373412] alg: No test for adler32 (adler32-zlib) [ 5356.387402] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 5356.625663] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 5358.319671] Key type lgssc registered [ 5359.571266] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5367.383449] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5367.409662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5368.907694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5373.091736] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5379.042820] LustreError: 172129:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5379.079789] LustreError: 172129:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5380.088842] LustreError: 172128:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5383.541174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5383.988181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5389.421210] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5398.739050] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5405.116035] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5412.718706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5413.099558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5415.168633] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5420.600496] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5428.784864] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5435.431187] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5435.472849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5435.497645] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5435.509160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.541157] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5436.871437] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5436.902391] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5436.932789] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5436.944258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.964822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5437.807176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.535902] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5438.544434] Lustre: Skipped 1 previous similar message [ 5438.568358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5438.575660] Lustre: Skipped 1 previous similar message [ 5438.577811] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5438.581558] Lustre: Skipped 1 previous similar message [ 5438.584924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.590994] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5438.595927] Lustre: Skipped 1 previous similar message [ 5439.821274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.968264] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5440.974102] Lustre: Skipped 1 previous similar message [ 5440.989725] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5441.003320] Lustre: Skipped 1 previous similar message [ 5441.010084] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5441.016284] Lustre: Skipped 1 previous similar message [ 5441.019594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.029908] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5441.046896] Lustre: Skipped 1 previous similar message [ 5442.243659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.356913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.158533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.044454] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5445.048825] Lustre: Skipped 3 previous similar messages [ 5445.078765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5445.103134] Lustre: Skipped 3 previous similar messages [ 5445.120177] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5445.123242] Lustre: Skipped 3 previous similar messages [ 5445.135646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.148118] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5445.153597] Lustre: Skipped 3 previous similar messages [ 5446.214400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.101990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.012137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.018231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.886626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.711563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.816306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.782442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.838966] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5453.845523] Lustre: Skipped 8 previous similar messages [ 5453.878714] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5453.896362] Lustre: Skipped 8 previous similar messages [ 5453.905865] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5453.916577] Lustre: Skipped 8 previous similar messages [ 5453.931302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.943907] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5453.953682] Lustre: Skipped 8 previous similar messages [ 5455.072597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.177817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.179793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.241327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.364613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.443167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.809837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.944431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.376527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.710724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.861364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.738951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.055669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.071478] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5470.079649] Lustre: Skipped 13 previous similar messages [ 5470.108528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5470.118222] Lustre: Skipped 13 previous similar messages [ 5470.122967] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5470.128866] Lustre: Skipped 13 previous similar messages [ 5470.134182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.154995] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5470.168568] Lustre: Skipped 13 previous similar messages [ 5471.261054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.364617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.507630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.534622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.576233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.428037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.343375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.329764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.346978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.667534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.265124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.180405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.031973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.919407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.596782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.351291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.130360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.891473] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5486.915186] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5486.917807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.672179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.471977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.124437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.141062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.993285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.538747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.130977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.747867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.334938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.955755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.699389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.580841] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5495.596598] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5495.599596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.673136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.833069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.608434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.607313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.563887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.496673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.100750] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5502.108165] Lustre: Skipped 36 previous similar messages [ 5502.124334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5502.130748] Lustre: Skipped 36 previous similar messages [ 5502.133653] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5502.136415] Lustre: Skipped 36 previous similar messages [ 5502.141446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.156437] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5502.161183] Lustre: Skipped 36 previous similar messages [ 5503.217200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.080666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.189422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.106491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.032862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.064880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.132420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.258454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.371844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.113054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.136533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.996861] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5514.004867] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5514.021376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.169494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.257584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.332133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.295527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.288345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.405211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.486776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.438902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.308611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.234343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.310388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.149221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.161805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.018187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.996301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.845994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.868807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.768906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.883991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.760245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.740067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.669844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.713862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.803236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.737428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.647564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.438625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.408933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.424783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.270457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.316591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.151923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.173653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.151371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.291751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.342753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.376397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.270228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.231356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.082516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.977160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.142051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.380403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.302825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.321231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.143595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.130861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.128475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.284256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.424065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.573959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.651868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.771657] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5566.794669] Lustre: Skipped 64 previous similar messages [ 5566.815783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5566.833268] Lustre: Skipped 64 previous similar messages [ 5566.842316] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5566.849201] Lustre: Skipped 64 previous similar messages [ 5566.858514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.873792] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5566.888370] Lustre: Skipped 64 previous similar messages [ 5567.894640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.762930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.876532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.806675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.805726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.872755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.967033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.001183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.145279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.968308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.908123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.804405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.772632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.695346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.645299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.835936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.792443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.679462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.613700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.572179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.418318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.369657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.303827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.249968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.232095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.232773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.216949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.308911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.552592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.602440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.601584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.462810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.285170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.926645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.755437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.591632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.484681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.582401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.520355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.276539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.961784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.798681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.788199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.568278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.258931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.029877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.762812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.830943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.949216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.102822] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5614.116680] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5614.127578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.154070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.153918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.009957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.329811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.225916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.468748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.671987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.849681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.516878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.279935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.416959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.294545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.702491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.980074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.080116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.915749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.856713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.721826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.624468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.499506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.433255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.301423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.319224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.383124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.264834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.240758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.338576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.300571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.446589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.398082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.351880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.192221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.162167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.109963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.237295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.073342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.117589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.164748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.274186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.238285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.205454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.035884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.708720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.448228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.600451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.612454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.803658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.955634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.100153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.138971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.294174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.190684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.063743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.103308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.018275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.035613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.965388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.924950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.839161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.226402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.220680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.332938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.397729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.588529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.593782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.845994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.875850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.804614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.881424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.886690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.561856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.482590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.524526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.580549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.445189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.637848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.458801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.395150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.492654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.443735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.439358] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5695.453945] Lustre: Skipped 130 previous similar messages [ 5695.473536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5695.486440] Lustre: Skipped 130 previous similar messages [ 5695.491841] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5695.498948] Lustre: Skipped 130 previous similar messages [ 5695.503505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.512895] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5695.527128] Lustre: Skipped 130 previous similar messages [ 5696.354982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.336708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.331932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.359615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.193252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.154855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.876540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.112665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.470979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.271310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.029363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.870887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.986398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.130233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.157728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.525248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.635297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.801739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.734480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.783192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.728387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.879817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.747662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.324991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.572194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.082572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.790829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.793884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.676379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.789641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.546848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.609125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.320573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.020593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.831108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.938960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.892122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.991580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.836612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.660528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.524162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.517738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.670912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.882389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.931891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.185174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.025698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.006040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.178373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.267282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.248494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.340590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.315885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.298935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.436787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.570255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.523959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.594355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.694505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.902255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.132903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.316585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.113843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.181472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.086503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.977400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.735208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.601731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.649626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.635431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.591048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.712496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.777951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.864276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.089111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.051332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.037839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.994462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.001214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.994763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.009592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.979770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.925051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.809526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.791435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.839787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.872132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.921202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.834623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.085250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.094580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.275774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.083716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.059690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.857890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.859354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.840175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.865190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.708744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.549569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.420943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.504796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.391448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.265034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.162405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.009429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.759692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.718364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.794790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.738517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.743638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.777626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.749224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.740550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.720392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.555512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.441375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.355326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.314876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.420127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.293856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.373685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.205595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.987177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.912973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.681992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.671555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.672864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.688264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.716170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.725502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.784085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.687798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.743826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.643155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.613862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.654029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.655201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.638672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.474065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.317593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.275327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.142594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.143058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.178379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.076472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.112817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.223321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.196989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.306503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.375702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.477442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.699979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.838844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.062979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.142572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.109408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.147797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.151186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.184176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.157511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.144906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.930723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.919360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.953389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.047882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.087298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.264244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.994231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.903708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.773986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.679472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.719442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.761478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.759366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.829762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.645323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.703115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.702209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.657177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.735170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.835485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.955356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.120767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.038777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.922175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.922482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.848529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.777485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.748185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.572988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.419971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.392259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.185585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.433452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.577543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.353978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.386388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.842960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.116782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.399563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.566742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.773371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.864095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.815995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.836250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.793287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.989088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.257622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.146320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.009639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.972922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.066435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.292279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.411730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.435992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.637129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.755281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.984767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.176733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.451635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.415103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.725157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.742302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.838087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.763899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.746671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.866278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.010073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.084893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.244527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.463308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.692845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.900091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.113544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.006571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.187144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.265236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.327148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.273361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.183889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.369130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.640722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.667748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.603773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.521689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.600567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.383696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.573294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.885731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.999883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.253234] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5952.266326] Lustre: Skipped 251 previous similar messages [ 5952.298344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5952.309713] Lustre: Skipped 251 previous similar messages [ 5952.319195] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5952.323775] Lustre: Skipped 251 previous similar messages [ 5952.333567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.345475] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5952.350707] Lustre: Skipped 251 previous similar messages [ 5953.458419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.524151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.976692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.486759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.691063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.066529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.643496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.411203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.727148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.250706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.373260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.703877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.040479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.448695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.725853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.961575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.041949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.150168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.258966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.996092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.856222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.555626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.744537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.869321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.137464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.202886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.542135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.673694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.482927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.686436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.467152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.460591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.358459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.195483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.258566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.429177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.529039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.633093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.691822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.763165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.995585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.164403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.262711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.209230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.225188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.276397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.346899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.249392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.256629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.086315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.123827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.154345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.145215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.094964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.258622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.039881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.137636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.281030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.214840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.224889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.231040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.179712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.245182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.333975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.359716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.241622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.196926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.088673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.131792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.502370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.798859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.153817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.244686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.226387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.188505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.128427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.021693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.914337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.892196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.776614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.880571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.783652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.824345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.725962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.503596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.309735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.204648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.152693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.881321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.903402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.695545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.580099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.568357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.461674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.431597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.475653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.343859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.436832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.269700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.209642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.196935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.146339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.139484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.099086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.952681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.822668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.894521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.893953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.772364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.905602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.948437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.946271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.050504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.159521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.200759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.374237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.372484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.316738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.298596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.435255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.367293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.308094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.064680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.143791] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6082.154742] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6082.156967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.259562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.209883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.165653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.992536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.016211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.898902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.847572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.771135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.853758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.916104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.860547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.791208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.705650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.562751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.683229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.749838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.658725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.584212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.933347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.940753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.040699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.095454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.941675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.868846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.761099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.605506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.624743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.632467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.697228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.643578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.632671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.541329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.519120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.405952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.470211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.592975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.661210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.599220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.643767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.542834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.489327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.622714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.658870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.726507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.958229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.010234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.050295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.054866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.274588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.514030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.543709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.648714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.764425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.749301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.749286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.840156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.806727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.021924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.045991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.927738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.964327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.109967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.199809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.184811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.195807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.144987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.377264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.579621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.557391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.696400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.634900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.598734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.517092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.662101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.470566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.711801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.926606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.923982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.233434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.445245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.517601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.667241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.744463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.938382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.117870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.311232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.360984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.327525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.257624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.359339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.471372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.607797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.546173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.788614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.870330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.991125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.109953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.142307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.353594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.332453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.413868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.571789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.482343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.654150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.810940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.999302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.094790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.933974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.971525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.090851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.122486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.186565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.290855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.450054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.560728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.623747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.525543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.581757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.796845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.832686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.739328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.738876] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6209.753229] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6209.765362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.653483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.444597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.266598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.142794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.996568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.808227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.832334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.865529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.897262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.945864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.799659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.874807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.749636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.786528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.538845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.565508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.678827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.549534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.480149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.586734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.604080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.711821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.810907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.793676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.868129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.908265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.972528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.962505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.017695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.126293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.142951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.049161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.100265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.080210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.223865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.349450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.548255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.553640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.685260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.725817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.913253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.025335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.952442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.995073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.947970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.867672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.815972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.633766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.395596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.550328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.423140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.334853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.299400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.280878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.289721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.514702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.772885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.892944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.096960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.025475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.373673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.382150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.539167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.726556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.760433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.241150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.552804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.607742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.062990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.321886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.166175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.581545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.843372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.171066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.388776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.400793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.423944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.242428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.256550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.310985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.440394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.690650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.756072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.833733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.050897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.217819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.594037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.744518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.134637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.143803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.386878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.408920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.408421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.282352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.320968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.087777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.405774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.381772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.427939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.497970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.544356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.726988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.836559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.035584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.031983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.130887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.372247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.480644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.321163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.301427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.215550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.178392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.493162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.560765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.512448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.471388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.430787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.359013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.356883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.454150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.632447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.602785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.771241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.738408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.944399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.961988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.013319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.898661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.771761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.971615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.887428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.928709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.759578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.832897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.097789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.219332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.149547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.183897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.380470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.518669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.504894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.571790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.638706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.713682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.473228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.429981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.450690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.391333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.359943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.402894] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6367.410717] LustreError: 172148:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6367.416186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.406959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.817597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.734937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.640436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.631434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.536364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.438705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.192872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.275088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.212091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.067928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.126163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.104805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.108646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.024089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.055838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.747277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.554226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.666725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.690248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.635937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.662510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.643067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.424952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.516612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.608920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.855682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.819822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.209491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.592365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.945459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.840863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.917610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.065851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.220287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.538834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.766702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.681336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.868432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.131903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.227826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.366824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.351476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.433685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.328609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.388439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.468056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.433529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.639555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.606930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.782563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.936038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.110287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.271573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.282192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.202776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.354296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.647660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.555331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.683662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.677577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.894329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.877684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.710899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.629743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.591658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.561912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.416458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.383284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.345608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.116701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.059579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.821653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.808733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.498339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.588211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.651754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.765847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.718285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.891724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.226560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.404855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.249678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.564066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.860542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.213586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.217939] LustreError: 173950:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6816.230789] LustreError: 172148:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7235.574799] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7235.591466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7235.610592] Lustre: Skipped 482 previous similar messages [ 7235.626041] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7241.186414] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7241.208629] Lustre: Skipped 3 previous similar messages [ 7242.192826] Lustre: server umount lustre-OST0000 complete [ 7246.316195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7251.424895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7251.434118] Lustre: Skipped 2 previous similar messages [ 7256.545931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7260.127311] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7260.561772] Lustre: server umount lustre-MDT0000 complete [ 7261.666494] LustreError: 172128:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7261.687517] LustreError: 172128:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7264.199789] LustreError: 172108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788690971 with bad export cookie 654094884614802334 [ 7264.204206] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7264.205839] LustreError: 172108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7264.399430] Lustre: server umount lustre-MDT0001 complete [ 7277.228073] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 7280.846482] Key type lgssc unregistered [ 7281.386394] LNet: 227354:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7281.404623] LNetError: 227354:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7281.434942] LNet: Removed LNI 192.168.201.118@tcp [ 7283.055424] Key type .llcrypt unregistered [ 7283.057625] Key type ._llcrypt unregistered [ 7312.551736] Key type ._llcrypt registered [ 7312.555404] Key type .llcrypt registered [ 7312.714248] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 7314.177375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7314.217332] alg: No test for adler32 (adler32-zlib) [ 7315.191102] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 7315.514402] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 7317.279181] Key type lgssc registered [ 7318.927648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7327.515327] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7327.540868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7329.159635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.592975] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7339.488351] LustreError: 228766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7339.519704] LustreError: 228766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7341.931175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7342.318157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7346.970709] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7354.451767] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7360.149922] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7366.572313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7366.900806] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7373.876486] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7375.112121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7381.625954] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7387.319207] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7389.251472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7389.261722] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7389.265913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7389.275602] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7389.276672] LustreError: 228786:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7393.772594] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7393.781240] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7393.792767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7394.870643] Lustre: lustre-OST0000: Not available for connect from 192.168.201.18@tcp (stopping) [ 7395.812070] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7397.482722] Lustre: server umount lustre-OST0000 complete [ 7409.319170] LustreError: 228786:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7431.834800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7432.195598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7436.283778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.298174] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7439.188877] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7445.550226] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.568141] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7448.028670] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7449.673666] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7456.005736] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98f504fbc800.ost_server_uuid 50 [ 7457.693641] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff98f504fbc800.ost_server_uuid in IDLE state after 0 sec [ 7465.956899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7465.962091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7465.981039] Lustre: Skipped 1 previous similar message [ 7465.981603] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7466.010387] Lustre: Skipped 3 previous similar messages [ 7469.340723] Lustre: server umount lustre-OST0000 complete [ 7475.684323] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7475.689754] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7475.717970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7479.527963] Lustre: server umount lustre-MDT0000 complete [ 7481.313766] LustreError: 228766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7481.336481] LustreError: 228766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7483.483345] LustreError: 228747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788691191 with bad export cookie 12209700355619150065 [ 7483.487554] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7483.497712] LustreError: 228747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7484.092970] Lustre: server umount lustre-MDT0001 complete [ 7496.833921] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 7500.618368] Key type lgssc unregistered [ 7500.993247] LNet: 233974:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7501.001959] LNetError: 233974:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7502.053976] LNet: Removed LNI 192.168.201.118@tcp [ 7503.160710] Key type .llcrypt unregistered [ 7503.162298] Key type ._llcrypt unregistered [ 7540.412742] Key type ._llcrypt registered [ 7540.415806] Key type .llcrypt registered [ 7540.508368] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 7541.450128] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7541.469869] alg: No test for adler32 (adler32-zlib) [ 7542.439404] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 7542.645397] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 7544.287185] Key type lgssc registered [ 7546.224102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7558.271259] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7558.313683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7560.030963] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7565.495697] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7570.400782] LustreError: 235701:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7570.421326] LustreError: 235701:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7575.300743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7575.525274] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7575.726868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7581.544914] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7585.331496] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7593.119853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7593.504096] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7601.939242] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7604.795855] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7612.281574] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7612.799584] Lustre: lustre-OST0001: new disk, initializing [ 7612.816438] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7612.830836] Lustre: 238010:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7613.050771] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7618.251371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7618.265370] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7618.503259] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7621.123236] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7634.824807] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7641.853213] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7657.444627] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7657.470713] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7657.506395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7659.494534] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7659.501586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7659.526026] Lustre: Skipped 2 previous similar messages [ 7659.538140] Lustre: Skipped 3 previous similar messages [ 7662.994762] Lustre: server umount lustre-MDT0000 complete [ 7664.628439] LustreError: 236831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7664.647762] LustreError: 236831:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7666.490211] LustreError: 235683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788691374 with bad export cookie 12876450717245102816 [ 7666.492691] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7666.498641] LustreError: 235683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7666.659596] Lustre: server umount lustre-MDT0001 complete [ 7682.300137] Lustre: server umount lustre-OST0000 complete [ 7686.123118] Lustre: 234961:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788691377/real 1788691377] req@ffff88ef4ae25f80 x1875578716262272/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788691393 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7686.170415] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7687.558528] Lustre: server umount lustre-OST0001 complete [ 7715.072149] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 7728.428354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7729.081603] LustreError: 241120:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7729.156169] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7734.249401] LustreError: 241121:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7734.561965] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7739.379546] LustreError: 241120:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7745.290115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7745.693705] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7750.766260] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7754.221792] Lustre: 242263:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7762.619635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7763.136263] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7771.353245] LustreError: 242618:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7771.365106] LustreError: 242618:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7771.376553] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7771.650299] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7774.480726] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7780.580918] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7786.010750] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7786.403634] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7793.876400] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7798.594171] Lustre: 244133:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7816.684803] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7816.689472] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7816.719382] Lustre: Skipped 3 previous similar messages [ 7816.737885] Lustre: Skipped 3 previous similar messages [ 7818.566520] Lustre: server umount mylustre-MDT0000 complete [ 7821.807440] LustreError: 241117:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7821.827090] LustreError: 241117:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7823.173280] LustreError: 244335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788691530 with bad export cookie 12876450717245105686 [ 7823.177972] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7823.180467] LustreError: 244335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7823.692227] Lustre: server umount mylustre-MDT0001 complete [ 7837.697639] Lustre: server umount mylustre-OST0000 complete [ 7852.652272] Lustre: server umount mylustre-OST0001 complete [ 7876.740596] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 7889.260923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7889.822156] LustreError: 246449:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7889.951277] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7889.966794] Lustre: Skipped 1 previous similar message [ 7895.479792] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7904.530886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7911.365318] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7915.035692] Lustre: 247591:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7922.730254] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7929.762800] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7931.325212] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7936.469706] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7938.720915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7944.192242] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7945.631483] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 7953.601563] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7957.581278] Lustre: 249462:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7966.176702] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7966.187985] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7966.207579] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7971.800145] Lustre: server umount tfs-MDT0000 complete [ 7974.891930] LustreError: 246450:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7974.938626] LustreError: 246450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 7976.839293] LustreError: 246425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788691684 with bad export cookie 12876450717245107856 [ 7976.857974] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7976.863782] LustreError: 246425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7977.396185] Lustre: server umount tfs-MDT0001 complete [ 7992.221593] Lustre: server umount tfs-OST0000 complete [ 8006.349701] Lustre: server umount tfs-OST0001 complete [ 8027.315285] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8037.405923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8038.241871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8038.248720] Lustre: Skipped 3 previous similar messages [ 8042.578943] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8050.656796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8056.785283] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8060.497891] Lustre: 252854:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8067.494510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8074.657577] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8076.296639] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 8083.449498] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8083.940916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8089.604106] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8091.532473] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8101.011869] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8105.213877] Lustre: 254726:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8113.134172] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:50:19 (1788691819) [ 8130.544661] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8130.550313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8130.565676] Lustre: Skipped 6 previous similar messages [ 8130.579324] Lustre: Skipped 7 previous similar messages [ 8144.351461] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8144.657915] Lustre: server umount lustre-MDT0000 complete [ 8145.906125] LustreError: 252445:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8145.942736] LustreError: 252445:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8149.039588] LustreError: 251688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788691856 with bad export cookie 12876450717245109774 [ 8149.047706] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8149.054094] LustreError: 251688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8151.009715] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8151.019772] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8151.028945] Lustre: Skipped 1 previous similar message [ 8151.046723] Lustre: Skipped 9 previous similar messages [ 8155.397988] Lustre: server umount lustre-MDT0001 complete [ 8165.813405] Lustre: server umount lustre-OST0000 complete [ 8176.531608] Lustre: server umount lustre-OST0001 complete [ 8183.646859] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 8193.517493] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8231.813314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8232.109198] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8232.154545] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8232.240899] Lustre: lustre-MDT0000: new disk, initializing [ 8232.353764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8232.357817] Lustre: Skipped 3 previous similar messages [ 8232.385975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8237.227471] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8248.547345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8248.696486] Lustre: 257914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8248.761371] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8248.768029] Lustre: Skipped 1 previous similar message [ 8248.849492] Lustre: lustre-MDT0001: new disk, initializing [ 8249.089834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8249.122601] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8253.851948] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8265.985813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8266.401421] Lustre: lustre-OST0000: new disk, initializing [ 8266.405506] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8266.412223] Lustre: 259494:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8267.581046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8267.591427] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8267.734453] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8273.745224] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8285.866584] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8286.034810] Lustre: lustre-OST0001: new disk, initializing [ 8286.038359] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8286.047618] Lustre: 260364:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8287.294577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8287.303122] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8287.370340] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8292.471229] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8304.744805] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:53:31 (1788692011) [ 8320.474274] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:53:46 (1788692026) [ 8338.935170] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8338.958088] Lustre: Skipped 1 previous similar message [ 8338.961604] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8338.968723] Lustre: Skipped 3 previous similar messages [ 8350.175613] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8350.251364] Lustre: server umount lustre-OST0000 complete [ 8360.629760] Lustre: server umount lustre-MDT0000 complete [ 8364.756861] LustreError: 259493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788692072 with bad export cookie 12876450717245111559 [ 8364.763722] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8364.773663] LustreError: 259493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8379.363229] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8379.565227] Lustre: server umount lustre-MDT0001 complete [ 8390.830771] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 8469.233318] Lustre: server umount lustre-OST0001 complete [ 8476.611471] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 8495.010434] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8534.771569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8535.083461] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8535.108328] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8535.170397] Lustre: lustre-MDT0000: new disk, initializing [ 8535.260864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8535.263733] Lustre: Skipped 3 previous similar messages [ 8535.273456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8539.919982] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8550.703550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8550.893452] Lustre: 265233:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8550.950422] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8550.954514] Lustre: Skipped 1 previous similar message [ 8551.067421] Lustre: lustre-MDT0001: new disk, initializing [ 8551.169764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8551.177546] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8556.770355] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8568.294954] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8574.505698] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8581.528898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8581.850577] Lustre: lustre-OST0000: new disk, initializing [ 8581.870944] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8581.881263] Lustre: 267200:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8583.531603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8583.543820] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8583.648579] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8587.693198] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8598.715336] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8609.250123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8609.269909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8609.272188] LustreError: Skipped 1 previous similar message [ 8609.273062] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8609.273068] Lustre: Skipped 12 previous similar messages [ 8609.327395] Lustre: Skipped 5 previous similar messages [ 8612.502737] Lustre: server umount lustre-OST0000 complete [ 8617.955606] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8622.668966] Lustre: server umount lustre-MDT0000 complete [ 8624.610077] LustreError: 267309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8624.638851] LustreError: 267309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 8626.438769] LustreError: 265225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788692333 with bad export cookie 12876450717245113029 [ 8626.451904] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8626.902276] Lustre: server umount lustre-MDT0001 complete [ 8638.311752] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 8641.861961] Key type lgssc unregistered [ 8642.260525] LNet: 268960:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8642.274326] LNetError: 268960:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8643.302575] LNet: Removed LNI 192.168.201.118@tcp [ 8644.847587] Key type .llcrypt unregistered [ 8644.854112] Key type ._llcrypt unregistered [ 8675.545704] Key type ._llcrypt registered [ 8675.548984] Key type .llcrypt registered [ 8675.664142] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8676.747121] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8676.861427] alg: No test for adler32 (adler32-zlib) [ 8678.310298] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 8678.598906] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 8680.271209] Key type lgssc registered [ 8681.770657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8689.899727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8689.923810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8691.560895] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8697.055688] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8701.923787] LustreError: 270426:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8701.936028] LustreError: 270426:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8707.061084] LustreError: 270425:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8707.947818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8708.468174] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8713.837154] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8723.189730] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8729.746793] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8735.885223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8736.170565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8737.217636] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8743.478953] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8753.732531] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8755.050686] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8757.728679] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8757.731542] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8757.763637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8757.791610] Lustre: Skipped 1 previous similar message [ 8759.857730] Lustre: server umount lustre-OST0000 complete [ 8764.896478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8764.908145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8764.946246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8764.956934] Lustre: Skipped 1 previous similar message [ 8767.968438] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8767.969625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8768.007850] Lustre: Skipped 1 previous similar message [ 8773.088942] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8773.098575] Lustre: Skipped 1 previous similar message [ 8778.208180] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8778.215388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8778.236328] Lustre: Skipped 1 previous similar message [ 8778.483195] Lustre: server umount lustre-MDT0000 complete [ 8782.776029] LustreError: 270406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788692490 with bad export cookie 6079705066453747490 [ 8782.784462] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8782.792436] LustreError: 270406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8783.143698] Lustre: server umount lustre-MDT0001 complete [ 8795.895965] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 8798.994204] Key type lgssc unregistered [ 8799.481723] LNet: 274022:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8799.492302] LNetError: 274022:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8799.505804] LNet: Removed LNI 192.168.201.118@tcp [ 8800.366411] Key type .llcrypt unregistered [ 8800.369695] Key type ._llcrypt unregistered [ 8820.646821] Key type ._llcrypt registered [ 8820.652534] Key type .llcrypt registered [ 8820.784810] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8822.534092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8822.748131] alg: No test for adler32 (adler32-zlib) [ 8824.072901] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 8824.634072] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 8826.431151] Key type lgssc registered [ 8828.591108] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8840.356391] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8840.402826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8856.031627] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8861.152199] LustreError: 275212:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.118@tcp: failed processing log, type 4: rc = -110 [ 8887.839190] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8894.700096] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 8902.726076] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8905.292692] Lustre: server umount lustre-OST0000 complete [ 8922.335581] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 8925.286949] Key type lgssc unregistered [ 8925.700670] LNet: 276841:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8925.710571] LNetError: 276841:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8926.759258] LNet: Removed LNI 192.168.201.118@tcp [ 8927.772405] Key type .llcrypt unregistered [ 8927.778794] Key type ._llcrypt unregistered [ 8972.620814] Key type ._llcrypt registered [ 8972.627891] Key type .llcrypt registered [ 8972.768919] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 8973.855408] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8974.023454] alg: No test for adler32 (adler32-zlib) [ 8975.207688] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 8975.555971] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 8977.311196] Key type lgssc registered [ 8979.093803] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8994.754224] loop0: detected capacity change from 0 to 204800000 [ 8995.048159] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8995.074592] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8995.642431] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8996.825397] loop0: detected capacity change from 0 to 204800000 [ 8997.080621] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8997.094351] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8997.481288] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8998.440966] loop0: detected capacity change from 0 to 204800000 [ 8998.672289] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8998.685344] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8998.847062] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9000.042424] loop0: detected capacity change from 0 to 204800000 [ 9000.375172] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9000.393700] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9000.612917] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9001.800174] loop0: detected capacity change from 0 to 204800000 [ 9001.826250] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9006.694851] loop0: detected capacity change from 0 to 204800000 [ 9006.732538] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9011.658479] loop0: detected capacity change from 0 to 204800000 [ 9011.747952] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9017.432369] loop0: detected capacity change from 0 to 204800000 [ 9017.470191] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9022.998784] loop0: detected capacity change from 0 to 204800000 [ 9023.097481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9023.182650] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9023.301794] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9023.310707] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9023.331427] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9023.351724] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9023.363781] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9023.379627] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9023.397737] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9023.425115] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9023.440996] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9023.476712] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9023.503839] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9023.529334] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9023.548633] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9023.564422] Lustre: 280254:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9029.991523] Lustre: server umount MGS complete [ 9031.090602] loop0: detected capacity change from 0 to 204800000 [ 9031.216085] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9031.851065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9031.914547] Lustre: lustre-MDT0000: Aborting client recovery [ 9032.886691] loop1: detected capacity change from 0 to 204800000 [ 9033.036301] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9033.194893] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9033.205932] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9033.226481] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9033.245054] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9033.257595] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9033.273825] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9033.297370] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9033.323781] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9033.565762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9033.679438] Lustre: lustre-MDT0001: Aborting client recovery [ 9034.838528] loop2: detected capacity change from 0 to 204800000 [ 9035.028994] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9035.184492] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9035.200565] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9035.230329] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9035.246523] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9035.254937] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9035.274579] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9035.292182] Lustre: 280767:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9035.450890] Lustre: Found index 0 for lustre-OST0000, updating log [ 9035.505803] Lustre: 280787:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9035.582944] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9035.619163] Lustre: lustre-OST0000: Aborting recovery [ 9036.579356] loop3: detected capacity change from 0 to 204800000 [ 9036.875196] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9037.034796] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9037.044285] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9037.066837] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9037.083980] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9037.100288] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9037.117537] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9037.126622] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9037.146191] Lustre: Found index 1 for lustre-OST0001, updating log [ 9037.179193] Lustre: 280858:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9037.256945] Lustre: lustre-OST0001: Aborting recovery [ 9042.353073] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9042.363673] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9042.392947] LustreError: 280630:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9042.464312] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9042.493038] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9043.364043] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9043.365959] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9047.403650] LustreError: 280597:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1719us: rc = -2 [ 9047.969133] LustreError: 280597:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 538us: rc = -2 [ 9047.993519] LustreError: 280597:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9063.905188] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9063.910223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9063.917133] Lustre: Skipped 2 previous similar messages [ 9063.948761] Lustre: Skipped 3 previous similar messages [ 9067.408835] Lustre: server umount lustre-MDT0000 complete [ 9068.592432] LustreError: 280579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788692776 with bad export cookie 5065851552776986565 [ 9068.608535] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9068.619905] LustreError: 280579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9069.024283] LustreError: 280596:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9069.028096] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9069.039304] LustreError: 280596:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9069.048559] Lustre: Skipped 1 previous similar message [ 9069.049227] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9073.056098] LustreError: 280865:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9073.058081] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9073.082839] LustreError: 280865:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9073.102440] Lustre: Skipped 2 previous similar messages [ 9075.148578] Lustre: server umount lustre-MDT0001 complete [ 9082.517135] Lustre: server umount lustre-OST0000 complete [ 9098.207363] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9098.467655] Lustre: server umount lustre-OST0001 complete [ 9108.825457] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 07:06:54 (1788692814) [ 9142.350432] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 9153.476310] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 9195.418089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9195.669207] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9195.708952] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9195.806206] Lustre: lustre-MDT0000: new disk, initializing [ 9195.882501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9195.885395] Lustre: Skipped 1 previous similar message [ 9195.922341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9201.135926] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9212.567673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9212.804769] Lustre: 284928:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9212.937799] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9212.947556] Lustre: Skipped 1 previous similar message [ 9213.139432] Lustre: lustre-MDT0001: new disk, initializing [ 9213.321293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9213.418448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9213.437318] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9219.475786] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9231.856603] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9238.644962] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9246.850900] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9247.330722] Lustre: lustre-OST0000: new disk, initializing [ 9247.338484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9247.346550] Lustre: 286899:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9247.554602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9252.984526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9253.002200] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9253.209741] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9254.861050] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9265.075535] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9271.442337] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9283.406733] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9287.649721] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9294.310214] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9330.146450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9330.150425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9330.160873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9330.163165] Lustre: Skipped 2 previous similar messages [ 9330.183019] LustreError: Skipped 1 previous similar message [ 9335.004669] Lustre: server umount lustre-OST0000 complete [ 9340.384436] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9340.388364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9340.410899] Lustre: Skipped 1 previous similar message [ 9340.424018] Lustre: Skipped 2 previous similar messages [ 9345.198186] Lustre: server umount lustre-MDT0000 complete [ 9345.511393] LustreError: 286092:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9345.539867] LustreError: 286092:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9349.607932] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9349.634068] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9349.729799] LustreError: 284921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788693057 with bad export cookie 5065851552776989267 [ 9349.751074] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9350.290861] Lustre: server umount lustre-MDT0001 complete [ 9360.413829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9365.932297] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9371.375707] Lustre: server umount MGS complete [ 9384.741513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9385.148096] LustreError: 290669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9385.211436] LustreError: 290682:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9385.283339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9390.283303] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9394.726330] LustreError: 290670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9394.753830] LustreError: 290670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9400.787309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9406.487559] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9415.068126] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9422.025615] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9430.517214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9431.121483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9431.141306] Lustre: Skipped 1 previous similar message [ 9438.554719] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9448.156690] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9486.822950] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9486.828996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9486.842619] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9486.856593] Lustre: Skipped 1 previous similar message [ 9487.936939] Lustre: server umount lustre-OST0000 complete [ 9493.984787] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9497.060608] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9497.079226] Lustre: Skipped 1 previous similar message [ 9499.969993] Lustre: server umount lustre-MDT0000 complete [ 9502.186785] LustreError: 290664:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9502.210527] LustreError: 290664:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9504.003504] LustreError: 290649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788693211 with bad export cookie 5065851552776991108 [ 9504.003955] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9504.008994] LustreError: 290649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9504.554913] Lustre: server umount lustre-MDT0001 complete [ 9520.078301] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 9523.114948] Key type lgssc unregistered [ 9523.518996] LNet: 294634:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9523.536195] LNetError: 294634:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9523.547242] LNet: Removed LNI 192.168.201.118@tcp [ 9524.802910] Key type .llcrypt unregistered [ 9524.804439] Key type ._llcrypt unregistered [ 9568.230773] Key type ._llcrypt registered [ 9568.232465] Key type .llcrypt registered [ 9568.389814] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 9588.869627] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 9589.808064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9589.934288] alg: No test for adler32 (adler32-zlib) [ 9591.038933] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [ 9591.271119] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 9592.927160] Key type lgssc registered [ 9593.879701] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9632.497539] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9632.541730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9633.758179] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9633.776757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9633.829392] Lustre: lustre-MDT0000: new disk, initializing [ 9633.888924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9633.903887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9639.690759] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9653.625470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9653.805203] Lustre: 298592:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9653.884154] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9653.891106] Lustre: Skipped 1 previous similar message [ 9654.011716] Lustre: lustre-MDT0001: new disk, initializing [ 9654.084049] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9654.140619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9654.163108] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9660.288063] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9669.668890] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9675.932747] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9682.649382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9682.858320] Lustre: lustre-OST0000: new disk, initializing [ 9682.862154] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9682.868641] Lustre: 300558:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9682.952342] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9689.221474] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9690.163301] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9690.178837] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9690.290854] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9698.946750] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9706.550261] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9719.485550] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9727.593292] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9739.794918] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9792.994609] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9793.005394] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9793.006629] LustreError: Skipped 1 previous similar message [ 9793.010233] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9793.037144] Lustre: Skipped 1 previous similar message [ 9798.132783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9798.151247] Lustre: Skipped 2 previous similar messages [ 9802.210255] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9802.600542] Lustre: server umount lustre-OST0000 complete [ 9807.842798] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9807.857976] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9807.872835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9813.575289] Lustre: server umount lustre-MDT0000 complete [ 9817.574865] LustreError: 302137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788693525 with bad export cookie 10200629920908425308 [ 9817.582859] LustreError: 302137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9817.583375] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9818.406715] Lustre: server umount lustre-MDT0001 complete [ 9828.760771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9834.038918] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9839.526708] Lustre: server umount MGS complete [ 9852.290335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9852.658272] LustreError: 304524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9852.680370] LustreError: 304524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9852.693419] LustreError: 304537:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9852.755488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9858.031479] LustreError: 304525:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9858.067215] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9863.139075] LustreError: 304524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9868.023904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9868.424692] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9873.592558] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9881.540789] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9888.879084] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9896.090906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9896.526035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9903.852960] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 9911.444304] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9938.918372] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9938.924872] Lustre: Skipped 1 previous similar message [ 9938.930752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9938.934640] Lustre: Skipped 2 previous similar messages [ 9939.784689] Lustre: server umount lustre-OST0000 complete [ 9944.036352] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9944.053870] Lustre: Skipped 2 previous similar messages [ 9944.057441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9944.067585] Lustre: Skipped 2 previous similar messages [ 9949.811025] Lustre: server umount lustre-MDT0000 complete [ 9953.798159] LustreError: 304506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788693661 with bad export cookie 10200629920908427268 [ 9953.813573] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9953.817989] LustreError: 304506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9954.240099] Lustre: server umount lustre-MDT0001 complete [ 9965.812567] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 9968.932125] Key type lgssc unregistered [ 9969.248605] LNet: 308466:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9969.253762] LNetError: 308466:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9970.282285] LNet: Removed LNI 192.168.201.118@tcp [ 9971.363948] Key type .llcrypt unregistered [ 9971.365677] Key type ._llcrypt unregistered [ 9993.023065] Key type ._llcrypt registered [ 9993.027024] Key type .llcrypt registered [ 9993.182703] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [10008.836670] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [10009.950936] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10010.018528] alg: No test for adler32 (adler32-zlib) [10011.057451] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [10011.278603] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [10012.943211] Key type lgssc registered [10014.141759] Lustre: Echo OBD driver; http://www.lustre.org/ [10041.195350] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [10056.341737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10056.384198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10057.653259] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10057.675834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10057.743839] Lustre: lustre-MDT0000: new disk, initializing [10057.879878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10057.907371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10062.400844] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [10071.911477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10072.165923] Lustre: lustre-OST0000: new disk, initializing [10072.169347] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10072.172292] Lustre: Skipped 1 previous similar message [10072.176476] Lustre: 312514:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10072.285665] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10076.542166] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10076.563892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10076.622564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10078.768363] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [10854.881152] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10854.890883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10858.109610] Lustre: server umount lustre-MDT0000 complete [10861.458619] LustreError: 311701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788694568 with bad export cookie 20768039434920912 [10861.462619] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10861.466856] LustreError: 311701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10861.633980] Lustre: server umount lustre-OST0000 complete [10884.734039] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [10894.258161] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [10922.750239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10922.922653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10922.947797] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10923.017649] Lustre: lustre-MDT0000: new disk, initializing [10923.058176] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10923.105328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10926.867422] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [10935.338612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10935.414200] Lustre: 316840:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10935.450494] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10935.453411] Lustre: Skipped 1 previous similar message [10935.503458] Lustre: lustre-MDT0001: new disk, initializing [10935.591402] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10935.629599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10935.639990] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10939.348911] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [10947.642810] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10951.967518] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10956.946457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10957.217768] Lustre: lustre-OST0000: new disk, initializing [10957.220970] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10957.226980] Lustre: 318807:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10957.315153] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10961.760129] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [10965.050559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10965.061191] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10965.146054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10969.530567] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10972.968791] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10973.163272] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10976.754684] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10976.940763] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10980.833656] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10980.834616] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10980.840256] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10980.841182] LustreError: Skipped 1 previous similar message [10980.872663] Lustre: Skipped 1 previous similar message [10984.842749] Lustre: server umount lustre-OST0000 complete [10991.073627] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10991.086780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10991.093347] Lustre: Skipped 2 previous similar messages [10993.984872] Lustre: server umount lustre-MDT0000 complete [10997.258419] LustreError: 316831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788694704 with bad export cookie 20768039437861724 [10997.261067] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10997.276694] LustreError: 316831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10997.521648] Lustre: server umount lustre-MDT0001 complete [11000.300573] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11001.557498] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:38:28 (1788694708) [11007.077060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11007.479215] LustreError: 321118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11007.506458] LustreError: 321118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11007.588821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11011.031071] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11012.580565] LustreError: 321119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11017.698552] LustreError: 321118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11017.915269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11018.201774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11022.181582] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11028.209839] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11032.985324] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11038.698688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11038.994103] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11045.248299] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11052.225586] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11059.024714] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11059.159346] Lustre: lustre-OST0001: new disk, initializing [11059.162235] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11059.167504] Lustre: 323952:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11059.245725] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11065.168832] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11067.447914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11067.454489] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11067.477658] LustreError: 324417:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11068.521821] LustreError: 324417:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11068.528529] LustreError: 324417:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11069.538073] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11071.605643] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11071.616041] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11074.609094] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11074.812548] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11075.746754] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11075.753212] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11079.780982] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98f508ddd000.ost_server_uuid 50 [11081.009290] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff98f508ddd000.ost_server_uuid in FULL state after 0 sec [11083.945043] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11083.958855] LustreError: 324412:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11086.493769] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98f508ddd000.ost_server_uuid 50 [11087.977949] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff98f508ddd000.ost_server_uuid in FULL state after 0 sec [11090.246793] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11103.208439] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11103.212197] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11103.214850] Lustre: Skipped 2 previous similar messages [11103.230652] Lustre: Skipped 1 previous similar message [11108.325294] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11108.329346] Lustre: Skipped 1 previous similar message [11117.023149] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11117.175071] Lustre: server umount lustre-OST0001 complete [11118.567752] LustreError: 322937:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11118.585101] LustreError: 322937:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11123.683321] LustreError: 324413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11123.684501] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11123.685923] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11123.685930] Lustre: Skipped 5 previous similar messages [11123.700414] LustreError: 324413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11123.736272] Lustre: Skipped 1 previous similar message [11127.429532] Lustre: server umount lustre-OST0000 complete [11130.849844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11133.924674] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11133.939554] Lustre: Skipped 1 previous similar message [11136.602056] Lustre: server umount lustre-MDT0000 complete [11139.042589] LustreError: 321845:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11139.070891] LustreError: 321845:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11139.888765] LustreError: 321099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788694847 with bad export cookie 20768039437862613 [11139.893125] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11139.895882] LustreError: 321099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11140.228865] Lustre: server umount lustre-MDT0001 complete [11149.026576] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [11151.394254] Key type lgssc unregistered [11151.678066] LNet: 326636:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11151.690728] LNetError: 326636:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11151.714067] LNet: Removed LNI 192.168.201.118@tcp [11152.429143] Key type .llcrypt unregistered [11152.431186] Key type ._llcrypt unregistered [11173.061656] Key type ._llcrypt registered [11173.063726] Key type .llcrypt registered [11173.158202] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11174.341210] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11174.363046] alg: No test for adler32 (adler32-zlib) [11175.476258] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [11175.695065] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [11177.335226] Key type lgssc registered [11178.470657] Lustre: Echo OBD driver; http://www.lustre.org/ [11184.549546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11184.574606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11185.910242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11188.522979] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11193.746986] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11196.390464] LustreError: 328034:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11196.401355] LustreError: 328034:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11199.319809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11199.605847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11203.376954] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11210.415628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11210.657940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11211.694984] LustreError: 329710:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11211.710334] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11215.295324] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11216.873291] LustreError: 329710:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11220.853541] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11221.998026] LustreError: 329710:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11225.815994] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11225.954285] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11231.219674] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11231.222822] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11237.412818] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11242.977822] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11243.098395] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11338.720903] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11338.730522] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11338.738154] Lustre: Skipped 1 previous similar message [11338.739290] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11338.745291] Lustre: Skipped 1 previous similar message [11341.780979] Lustre: server umount lustre-OST0001 complete [11343.854047] LustreError: 331070:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11343.869852] LustreError: 331070:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11347.943834] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11347.949647] LustreError: Skipped 1 previous similar message [11347.953053] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11347.980613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11348.961523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11351.512322] Lustre: server umount lustre-OST0000 complete [11354.084681] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11354.102044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11354.107460] Lustre: Skipped 2 previous similar messages [11359.205158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11359.209787] Lustre: Skipped 1 previous similar message [11360.342899] Lustre: server umount lustre-MDT0000 complete [11363.304855] LustreError: 328014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788695070 with bad export cookie 4952032425383141344 [11363.307476] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11363.311993] LustreError: 328014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11363.602490] Lustre: server umount lustre-MDT0001 complete [11371.876738] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [11373.838748] Key type lgssc unregistered [11374.045608] LNet: 332973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11374.051496] LNetError: 332973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11374.066770] LNet: Removed LNI 192.168.201.118@tcp [11374.669690] Key type .llcrypt unregistered [11374.671717] Key type ._llcrypt unregistered [11405.697525] Key type ._llcrypt registered [11405.699540] Key type .llcrypt registered [11405.797925] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11406.571603] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11406.588677] alg: No test for adler32 (adler32-zlib) [11407.638847] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [11407.924128] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [11409.567438] Key type lgssc registered [11410.395260] Lustre: Echo OBD driver; http://www.lustre.org/ [11417.489620] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11417.507808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11418.777476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11421.957455] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11424.228522] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11424.241391] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11428.422479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11428.725247] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11431.526561] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11433.468651] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11437.745794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11438.096217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11439.139732] LustreError: 336726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11442.511973] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11444.199641] LustreError: 336726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11445.238966] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11448.397821] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11448.543150] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11452.567334] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11453.939693] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11460.110959] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11471.515562] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11479.524466] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11479.528970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11479.536138] Lustre: Skipped 2 previous similar messages [11479.542633] Lustre: Skipped 1 previous similar message [11484.642281] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11484.778807] Lustre: server umount lustre-MDT0000 complete [11487.352927] LustreError: 335218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788695194 with bad export cookie 985822429185332836 [11487.353257] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11487.362708] LustreError: 335218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11487.586476] Lustre: server umount lustre-MDT0001 complete [11500.587813] Lustre: server umount lustre-OST0000 complete [11513.682699] Lustre: server umount lustre-OST0001 complete [11525.459251] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11532.689558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11532.952916] LustreError: 340497:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11532.970040] LustreError: 340497:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11533.029740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11536.367410] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11538.406375] LustreError: 340498:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11538.414950] LustreError: 340498:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11541.848642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11542.109414] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11544.793547] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11546.868971] Lustre: 341637:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11551.477452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11552.752434] LustreError: 341993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11555.818496] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11557.869866] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11562.407950] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11562.639770] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11562.646599] Lustre: Skipped 1 previous similar message [11567.603681] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11568.117972] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11573.567497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11576.660471] Lustre: 343506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11582.944225] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11582.958691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11582.977445] Lustre: Skipped 1 previous similar message [11582.981469] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11582.985657] Lustre: Skipped 3 previous similar messages [11586.038705] Lustre: server umount lustre-MDT0000 complete [11588.582770] LustreError: 340493:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11588.604670] LustreError: 340493:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11588.959838] LustreError: 340480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788695296 with bad export cookie 985822429185334544 [11588.971685] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11589.316218] Lustre: server umount lustre-MDT0001 complete [11601.972994] Lustre: server umount lustre-OST0000 complete [11614.985580] Lustre: server umount lustre-OST0001 complete [11617.451858] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11618.618677] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11619.848723] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:48:46 (1788695326) [11635.334967] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11641.426352] loop0: detected capacity change from 0 to 1099511627776 [11641.817228] SGI XFS with ACLs, security attributes, quota, no debug enabled [11641.821535] *** VALIDATE xfs *** [11641.827195] XFS (loop0): Mounting V5 Filesystem [11641.853876] XFS (loop0): Ending clean mount [11642.713784] loop1: detected capacity change from 0 to 18691697672192 [11695.517853] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [11702.099391] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11726.818892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11726.982347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11727.001947] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11727.055334] Lustre: lustre-MDT0000: new disk, initializing [11727.102477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11727.114585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11729.880272] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11737.216940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11737.321071] Lustre: 348630:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11737.376389] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11737.378918] Lustre: Skipped 1 previous similar message [11737.438577] Lustre: lustre-MDT0001: new disk, initializing [11737.544578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11737.559421] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11740.278621] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11746.900886] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11750.157066] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11754.424598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11754.591707] Lustre: lustre-OST0000: new disk, initializing [11754.594476] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11754.599103] Lustre: 350596:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11756.114475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11756.125092] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11756.176860] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11758.447900] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11765.006276] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11768.150715] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11768.341572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11771.347371] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11771.477334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11776.997045] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11776.997459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11777.012105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11777.013831] Lustre: Skipped 4 previous similar messages [11777.022384] Lustre: Skipped 5 previous similar messages [11782.118275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11782.127021] Lustre: Skipped 1 previous similar message [11787.231136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11787.341243] Lustre: server umount lustre-OST0000 complete [11792.355570] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11792.367500] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11792.377410] Lustre: Skipped 3 previous similar messages [11796.090172] Lustre: server umount lustre-MDT0000 complete [11797.472290] LustreError: 348642:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11797.485065] LustreError: 348642:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11798.470563] LustreError: 348624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788695505 with bad export cookie 985822429185336091 [11798.475519] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11798.478743] LustreError: 348624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [11798.699080] Lustre: server umount lustre-MDT0001 complete [11803.860375] XFS (loop0): Unmounting Filesystem [11806.728810] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:51:53 (1788695513) [11810.544968] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11810.833549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11810.839187] Lustre: Skipped 2 previous similar messages [11813.492220] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11818.499940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11821.030747] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11825.393760] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11829.163795] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11833.483342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11837.265361] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11841.651269] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11854.310576] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11854.317316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11854.323586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11854.324583] Lustre: Skipped 2 previous similar messages [11857.442615] Lustre: server umount lustre-OST0000 complete [11864.545459] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11864.558774] Lustre: Skipped 1 previous similar message [11866.568545] Lustre: server umount lustre-MDT0000 complete [11869.102812] LustreError: 353136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788695576 with bad export cookie 985822429185336973 [11869.104100] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11869.114552] LustreError: 353136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11869.333223] Lustre: server umount lustre-MDT0001 complete [11877.029534] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [11878.954902] Key type lgssc unregistered [11879.191642] LNet: 356777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11879.199368] LNetError: 356777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11879.214772] LNet: Removed LNI 192.168.201.118@tcp [11879.846643] Key type .llcrypt unregistered [11879.849613] Key type ._llcrypt unregistered [11898.056388] Key type ._llcrypt registered [11898.057712] Key type .llcrypt registered [11898.113148] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [11898.838559] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11898.853184] alg: No test for adler32 (adler32-zlib) [11899.817358] Lustre: Lustre: Build Version: 2.17.58_2_g2f2534e [11899.965201] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [11901.567502] Key type lgssc registered [11902.230961] Lustre: Echo OBD driver; http://www.lustre.org/ [11907.031332] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11907.049505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11908.337374] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11910.943248] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11916.346515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11916.658416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11919.383672] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11923.711519] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11927.353714] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11931.562677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11931.794512] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11935.544590] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11940.181533] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11947.492312] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11947.498866] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11947.502341] Lustre: Skipped 1 previous similar message [11948.220958] Lustre: server umount lustre-MDT0001 complete [11952.611747] LustreError: 358318:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11952.621958] LustreError: 358318:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11953.767566] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11953.825375] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11953.831348] Lustre: Found index 1 for lustre-MDT0001, updating log [11953.857430] Lustre: 358307:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11954.012106] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11956.497517] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [11959.274291] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11960.312090] LustreError: 361917:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11960.317070] LustreError: 361917:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11960.323867] LustreError: 358311:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11960.357750] LustreError: 361920:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -17 [11960.363158] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11960.363158] [12259.869707] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [12259.872116] Lustre: Skipped 1 previous similar message [12261.346753] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12261.347633] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12261.349105] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12261.354190] Lustre: Skipped 2 previous similar messages [12264.980890] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [12264.983472] Lustre: Skipped 1 previous similar message [12265.052793] Lustre: server umount lustre-MDT0001 complete [12266.465775] LustreError: 358315:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12266.476398] LustreError: 358315:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12270.088166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12270.101466] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12270.148363] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12270.151061] Lustre: Found index 1 for lustre-MDT0001, updating log [12270.170574] Lustre: 358305:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12270.317342] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12272.294764] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [12275.698129] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12276.723890] LustreError: 372854:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12276.727586] LustreError: 372854:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12276.733652] LustreError: 358311:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12276.736182] LustreError: 358311:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12276.792469] LustreError: 372861:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12276.796716] LustreError: 372861:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12276.799596] LustreError: 372861:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -17 [12276.804250] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.118@tcp [12276.804250] [12577.319545] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [12577.763895] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12577.764882] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12577.771202] Lustre: Skipped 1 previous similar message [12587.545705] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [12587.552258] Lustre: Skipped 5 previous similar messages [12589.023127] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12589.092243] Lustre: server umount lustre-MDT0001 complete [12592.662244] LustreError: 358313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12594.042920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12594.093473] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12594.095812] Lustre: Found index 1 for lustre-MDT0001, updating log [12594.113761] Lustre: 358308:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12594.269160] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12596.308326] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [12599.790803] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12600.813093] LustreError: 384186:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12600.823091] LustreError: 384186:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12600.827439] LustreError: 358311:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12600.831747] LustreError: 358311:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12600.880596] LustreError: 384189:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12600.889518] LustreError: 384189:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -114 [12600.894053] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12600.894053] [12911.585748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12911.588631] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12911.598841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12911.603039] Lustre: Skipped 2 previous similar messages [12917.709353] Lustre: server umount lustre-MDT0000 complete [12919.620121] LustreError: 358298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788696627 with bad export cookie 6926979044753799394 [12919.621590] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12919.626017] LustreError: 358298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12922.335971] LustreError: 358315:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12922.337170] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12922.351139] LustreError: 358315:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12922.362112] Lustre: Skipped 2 previous similar messages [12925.840324] Lustre: server umount lustre-MDT0001 complete [12927.855437] Lustre: server umount lustre-OST0000 complete [12932.795553] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [12937.846192] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [12954.493368] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:11:01 (1788696661) [12957.388666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12957.482573] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12957.496430] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12957.531964] Lustre: lustre-MDT0000: new disk, initializing [12957.557484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12957.564556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12959.108476] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all