[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 478707277 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003317] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008323] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011124] LSM: Security Framework initializing [ 0.012051] Yama: becoming mindful. [ 0.013032] SELinux: Initializing. [ 0.014047] *** VALIDATE selinux *** [ 0.022561] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027064] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028163] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029102] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030108] *** VALIDATE tmpfs *** [ 0.032256] *** VALIDATE proc *** [ 0.033232] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.035256] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037072] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039027] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.043235] debug: unmapping init [mem 0xffffffffb7e59000-0xffffffffb7e60fff] [ 0.045928] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046672] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047021] ... version: 2 [ 0.048011] ... bit width: 48 [ 0.049012] ... generic registers: 4 [ 0.050010] ... value mask: 0000ffffffffffff [ 0.051013] ... max period: 00007fffffffffff [ 0.052012] ... fixed-purpose events: 3 [ 0.053010] ... event mask: 000000070000000f [ 0.054298] rcu: Hierarchical SRCU implementation. [ 0.057398] smp: Bringing up secondary CPUs ... [ 0.058704] x86: Booting SMP configuration: [ 0.059034] .... node #0, CPUs: #1 #2 #3 [ 0.062373] smp: Brought up 1 node, 4 CPUs [ 0.064015] smpboot: Max logical packages: 1 [ 0.065017] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.143000] node 0 deferred pages initialised in 75ms [ 0.147112] devtmpfs: initialized [ 0.148337] x86/mm: Memory block size: 128MB [ 0.151884] gcov: version magic: 0x41383552 [ 0.154380] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.159079] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.162312] pinctrl core: initialized pinctrl subsystem [ 0.164163] [ 0.164759] ************************************************************* [ 0.167010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169011] ** ** [ 0.172010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.173010] ** ** [ 0.176011] ** This means that this kernel is built to expose internal ** [ 0.178008] ** IOMMU data structures, which may compromise security on ** [ 0.181014] ** your system. ** [ 0.183011] ** ** [ 0.186013] ** If you see this message and you are not debugging the ** [ 0.188011] ** kernel, report this immediately to your vendor! ** [ 0.191009] ** ** [ 0.193011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.195010] ************************************************************* [ 0.197677] NET: Registered protocol family 16 [ 0.199434] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.202057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.205059] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.208435] cpuidle: using governor menu [ 0.210490] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.214546] PCI: Using configuration type 1 for base access [ 0.217128] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.226057] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.227049] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.229112] cryptd: max_cpu_qlen set to 1000 [ 0.232285] ACPI: Added _OSI(Module Device) [ 0.234014] ACPI: Added _OSI(Processor Device) [ 0.235056] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.237010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.242319] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.249473] ACPI: Interpreter enabled [ 0.251066] ACPI: PM: (supports S0 S3 S4 S5) [ 0.253030] ACPI: Using IOAPIC for interrupt routing [ 0.256247] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.260592] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.272505] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.275037] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.278020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.281103] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.287594] acpiphp: Slot [2] registered [ 0.289249] acpiphp: Slot [5] registered [ 0.291106] acpiphp: Slot [6] registered [ 0.293191] acpiphp: Slot [7] registered [ 0.294153] acpiphp: Slot [8] registered [ 0.296155] acpiphp: Slot [9] registered [ 0.298161] acpiphp: Slot [10] registered [ 0.299140] acpiphp: Slot [3] registered [ 0.301125] acpiphp: Slot [4] registered [ 0.302117] acpiphp: Slot [11] registered [ 0.304119] acpiphp: Slot [12] registered [ 0.305068] acpiphp: Slot [13] registered [ 0.306130] acpiphp: Slot [14] registered [ 0.308201] acpiphp: Slot [15] registered [ 0.310164] acpiphp: Slot [16] registered [ 0.312128] acpiphp: Slot [17] registered [ 0.314190] acpiphp: Slot [18] registered [ 0.316161] acpiphp: Slot [19] registered [ 0.317128] acpiphp: Slot [20] registered [ 0.319189] acpiphp: Slot [21] registered [ 0.321108] acpiphp: Slot [22] registered [ 0.323118] acpiphp: Slot [23] registered [ 0.325291] acpiphp: Slot [24] registered [ 0.327103] acpiphp: Slot [25] registered [ 0.328086] acpiphp: Slot [26] registered [ 0.329121] acpiphp: Slot [27] registered [ 0.330097] acpiphp: Slot [28] registered [ 0.332093] acpiphp: Slot [29] registered [ 0.334101] acpiphp: Slot [30] registered [ 0.335306] acpiphp: Slot [31] registered [ 0.337076] PCI host bridge to bus 0000:00 [ 0.338018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.340022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.343022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.346024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.348038] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.352044] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.354190] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.357197] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.361493] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.371823] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.377516] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.380040] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.383029] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.385024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.389142] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.393787] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.396043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.399866] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.406014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.420945] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.426015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.433767] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.443024] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.454030] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.476026] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.487957] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.502017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.514024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.538018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.547835] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.558023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.565022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.592016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.604347] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.622014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.638015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.658015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.669999] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.681015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.696015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.718015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.732094] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.741015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.749015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.771015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.788298] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.791381] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.793374] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.797429] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.799244] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.805091] iommu: Default domain type: Passthrough [ 0.808423] SCSI subsystem initialized [ 0.810233] ACPI: bus type USB registered [ 0.812110] usbcore: registered new interface driver usbfs [ 0.814078] usbcore: registered new interface driver hub [ 0.816075] usbcore: registered new device driver usb [ 0.818186] pps_core: LinuxPPS API ver. 1 registered [ 0.820013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.823063] PTP clock support registered [ 0.826069] EDAC MC: Ver: 3.0.0 [ 0.827520] PCI: Using ACPI for IRQ routing [ 0.829849] NetLabel: Initializing [ 0.831013] NetLabel: domain hash size = 128 [ 0.833010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.835103] NetLabel: unlabeled traffic allowed by default [ 0.837100] vgaarb: loaded [ 0.839266] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.840014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.852303] clocksource: Switched to clocksource kvm-clock [ 0.972833] VFS: Disk quotas dquot_6.6.0 [ 0.975319] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.978929] *** VALIDATE ramfs *** [ 0.980759] *** VALIDATE hugetlbfs *** [ 0.982621] pnp: PnP ACPI init [ 0.984576] pnp: PnP ACPI: found 6 devices [ 1.001990] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.005382] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.007882] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.010550] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.013231] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.015597] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.018790] NET: Registered protocol family 2 [ 1.021134] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.026181] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.029809] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.035853] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.039902] TCP: Hash tables configured (established 65536 bind 65536) [ 1.043468] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.047281] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.050744] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054217] NET: Registered protocol family 1 [ 1.057024] RPC: Registered named UNIX socket transport module. [ 1.060605] RPC: Registered udp transport module. [ 1.063289] RPC: Registered tcp transport module. [ 1.065732] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.068672] NET: Registered protocol family 44 [ 1.070987] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.073592] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.076254] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.079106] PCI: CLS 0 bytes, default 64 [ 1.080929] Unpacking initramfs... [ 2.488050] debug: unmapping init [mem 0xffff8d47fcc54000-0xffff8d47fffbffff] [ 2.492512] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.496439] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.499403] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.999075] Initialise system trusted keyrings [ 3.002483] Key type blacklist registered [ 3.004783] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.015211] zbud: loaded [ 3.018502] *** VALIDATE nfs *** [ 3.019854] *** VALIDATE nfs4 *** [ 3.021824] pstore: using deflate compression [ 3.025791] Platform Keyring initialized [ 3.126724] NET: Registered protocol family 38 [ 3.128499] Key type asymmetric registered [ 3.129679] Asymmetric key parser 'x509' registered [ 3.131685] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.135235] io scheduler mq-deadline registered [ 3.136910] io scheduler kyber registered [ 3.138868] io scheduler bfq registered [ 3.140522] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.144130] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.147215] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.150677] ACPI: Power Button [PWRF] [ 3.156484] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.166124] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.178992] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.186144] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.204486] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.234042] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.263666] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.270858] Non-volatile memory driver v1.3 [ 3.274235] Linux agpgart interface v0.103 [ 3.313714] virtio_blk virtio1: [vda] 146144 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.316385] vda: detected capacity change from 0 to 74825728 [ 3.337345] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.340755] vdb: detected capacity change from 0 to 1073741824 [ 3.358481] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.361917] vdc: detected capacity change from 0 to 2621440000 [ 3.376897] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.380631] vdd: detected capacity change from 0 to 2621440000 [ 3.397407] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.400215] vde: detected capacity change from 0 to 4294967296 [ 3.417970] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.421559] vdf: detected capacity change from 0 to 4294967296 [ 3.428805] libphy: Fixed MDIO Bus: probed [ 3.437225] usbcore: registered new interface driver usbserial_generic [ 3.440245] usbserial: USB Serial support registered for generic [ 3.442868] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.447551] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.449777] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.453305] mousedev: PS/2 mouse device common for all mice [ 3.456510] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.457323] rtc_cmos 00:05: RTC can wake from S4 [ 3.465802] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.466058] rtc_cmos 00:05: registered as rtc0 [ 3.470304] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.471262] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.477030] intel_pstate: CPU model not supported [ 3.480367] hid: raw HID events driver (C) Jiri Kosina [ 3.482790] usbcore: registered new interface driver usbhid [ 3.484935] usbhid: USB HID core driver [ 3.486622] drop_monitor: Initializing network drop monitor service [ 3.488964] Initializing XFRM netlink socket [ 3.490770] NET: Registered protocol family 10 [ 3.494704] Segment Routing with IPv6 [ 3.496155] NET: Registered protocol family 17 [ 3.499428] mpls_gso: MPLS GSO support [ 3.504755] RAS: Correctable Errors collector initialized. [ 3.507165] AVX version of gcm_enc/dec engaged. [ 3.509456] AES CTR mode by8 optimization enabled [ 3.580796] sched_clock: Marking stable (3580733921, 0)->(4491592483, -910858562) [ 3.583519] registered taskstats version 1 [ 3.586688] Loading compiled-in X.509 certificates [ 3.590382] zswap: loaded using pool lzo/zbud [ 3.619793] Key type big_key registered [ 3.632390] Key type encrypted registered [ 3.634448] ima: No TPM chip found, activating TPM-bypass! [ 3.636823] ima: Allocated hash algorithm: sha1 [ 3.638173] ima: No architecture policies found [ 3.639525] evm: Initialising EVM extended attributes: [ 3.641191] evm: security.selinux [ 3.642420] evm: security.ima [ 3.643337] evm: security.capability [ 3.644477] evm: HMAC attrs: 0x1 [ 3.646662] rtc_cmos 00:05: setting system clock to 2026-08-24 23:19:36 UTC (1787613576) [ 3.654994] debug: unmapping init [mem 0xffffffffb8e03000-0xffffffffb8ffffff] [ 3.658979] debug: unmapping init [mem 0xffffffffb7b82000-0xffffffffb7e58fff] [ 3.666202] Write protecting the kernel read-only data: 28672k [ 3.671140] debug: unmapping init [mem 0xffffffffb6203000-0xffffffffb63fffff] [ 3.673964] debug: unmapping init [mem 0xffffffffb6b14000-0xffffffffb6bfffff] [ 3.713691] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.723670] systemd[1]: Detected virtualization kvm. [ 3.726228] systemd[1]: Detected architecture x86-64. [ 3.729325] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.755161] systemd[1]: No hostname configured. [ 3.757289] systemd[1]: Set hostname to . [ 3.759454] random: systemd: uninitialized urandom read (16 bytes read) [ 3.762132] systemd[1]: Initializing machine ID from random generator. [ 3.924091] random: systemd: uninitialized urandom read (16 bytes read) [ 3.927036] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.934461] random: systemd: uninitialized urandom read (16 bytes read) [ 3.937982] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.943660] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.646646] device-mapper: uevent: version 1.0.3 [ 4.649834] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. [ 5.174676] random: fast init done Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.615687] virtio_net virtio0 ens2: renamed from eth0 [ 5.749604] scsi host0: ata_piix [ 5.786598] scsi host1: ata_piix [ 5.788597] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.791673] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.151333] random: crng init done [ 10.152202] random: 7 urandom warning(s) missed due to ratelimiting [ 10.608227] dracut-initqueue[591]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 11.765674] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.899229] printk: systemd: 26 output lines suppressed due to ratelimiting [ 14.439854] hrtimer: interrupt took 5343733 ns [ 14.565651] SELinux: Disabled at runtime. [ 14.673359] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 14.689068] systemd[1]: Detected virtualization kvm. [ 14.692262] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 15.646252] systemd[1]: initrd-switch-root.service: Succeeded. [ 15.654584] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 15.668329] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 15.684821] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 15.694923] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 15.719417] systemd[1]: Starting Journal Service... Starting Journal Service... [ 15.739958] systemd[1]: Activating swap /dev/disk/by-label/SWAP... Activating swap /dev/disk/by-label/SWAP... Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 15.826945] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 16.936347] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 17.850540] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 17.963465] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 18.363365] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 18.426977] EDAC sbridge: Ver: 1.1.2 [ 21.717112] Key type dns_resolver registered [ 22.432510] NFS: Registering the id_resolver key type [ 22.436167] Key type id_resolver registered [ 22.438542] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started Login Service. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg316-server login: [ 103.050387] libcfs: loading out-of-tree module taints kernel. [ 103.122434] Key type ._llcrypt registered [ 103.128791] Key type .llcrypt registered [ 103.293092] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 127.181652] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 129.350365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 129.363530] alg: No test for adler32 (adler32-zlib) [ 131.070342] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 132.321047] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 134.128225] Key type lgssc registered [ 136.475627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 154.034354] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 195.317882] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 208.473523] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 208.516976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 209.739840] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 209.783267] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 209.889849] Lustre: lustre-MDT0000: new disk, initializing [ 209.972182] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 209.990654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.191329] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 228.608832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 228.730465] Lustre: 6519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 228.790690] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 228.795479] Lustre: Skipped 1 previous similar message [ 228.946515] Lustre: lustre-MDT0001: new disk, initializing [ 229.008699] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 229.055543] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 229.076818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 234.031670] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 238.841484] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 248.679567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 248.975799] Lustre: lustre-OST0000: new disk, initializing [ 248.979440] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 248.986475] Lustre: 8455:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 249.103984] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 255.662178] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.037197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 256.052491] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.193753] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 269.098454] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 269.250224] Lustre: lustre-OST0001: new disk, initializing [ 269.254703] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 269.261039] Lustre: 9528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 269.325472] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 275.891405] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 276.031477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 276.040576] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 276.140464] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 288.148777] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 299.405582] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 305.753829] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing check_logdir /tmp/testlogs/ [ 310.814373] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing yml_node [ 315.315379] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 317.692221] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 320.070352] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 321.824532] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 19:24:52 EDT 2026 [ 337.723161] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 339.226590] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 357.367805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 357.376497] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 357.395533] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 358.374191] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 358.375707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 358.401547] Lustre: Skipped 1 previous similar message [ 358.428416] Lustre: Skipped 3 previous similar messages [ 361.546383] Lustre: server umount lustre-MDT0000 complete [ 365.212802] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787613938 with bad export cookie 7527021833172383223 [ 365.219437] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 365.226039] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 368.615897] LustreError: 6527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 368.618475] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 368.634206] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 368.652625] LustreError: 6527:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 368.662320] Lustre: Skipped 2 previous similar messages [ 370.659345] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 370.661862] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 370.672270] Lustre: Skipped 2 previous similar messages [ 370.708319] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 371.632715] Lustre: server umount lustre-MDT0001 complete [ 375.792573] Lustre: server umount lustre-OST0000 complete [ 379.892616] Lustre: server umount lustre-OST0001 complete [ 386.928227] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 396.378226] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 434.615159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 434.863170] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 434.893839] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 434.994174] Lustre: lustre-MDT0000: new disk, initializing [ 435.103408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 435.138261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 439.899757] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 451.095834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 451.296791] Lustre: 15083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 451.373542] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 451.381368] Lustre: Skipped 1 previous similar message [ 451.493622] Lustre: lustre-MDT0001: new disk, initializing [ 451.602050] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 451.680806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 451.694497] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 457.529747] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 467.869272] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 474.549287] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 482.738319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 483.035880] Lustre: lustre-OST0000: new disk, initializing [ 483.039308] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 483.047843] Lustre: 17053:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 483.160590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 484.257691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 484.273146] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 484.353491] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 489.704656] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 500.258331] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 504.977240] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 505.160124] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 508.926236] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 509.148396] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 515.052955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 515.053975] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 515.058919] LustreError: Skipped 1 previous similar message [ 515.060322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 515.080790] Lustre: Skipped 1 previous similar message [ 517.607652] Lustre: server umount lustre-OST0000 complete [ 523.234238] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 523.239150] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 523.250520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 523.255808] Lustre: Skipped 1 previous similar message [ 527.744747] Lustre: server umount lustre-MDT0000 complete [ 530.404276] LustreError: 15089:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 530.418546] LustreError: 15089:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 531.675557] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614104 with bad export cookie 7527021833172384756 [ 531.676054] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 531.692327] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 532.248157] Lustre: server umount lustre-MDT0001 complete [ 538.221801] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:28:29 (1787614109) [ 544.641195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 545.098233] LustreError: 19319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.149708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 549.331365] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 550.370632] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.396703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 562.353919] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 570.755543] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 576.868293] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 583.250693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 590.319359] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 598.622018] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 605.561341] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 613.347642] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 613.352085] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 613.375119] Lustre: Skipped 3 previous similar messages [ 613.394846] Lustre: Skipped 4 previous similar messages [ 618.657564] Lustre: server umount lustre-MDT0000 complete [ 622.578609] LustreError: 21161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614195 with bad export cookie 7527021833172385638 [ 622.583638] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 622.589616] LustreError: 21161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 622.988460] Lustre: server umount lustre-MDT0001 complete [ 637.288217] Lustre: server umount lustre-OST0000 complete [ 658.841803] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 669.857887] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 670.294660] LustreError: 23702:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 670.331906] LustreError: 23702:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 670.407903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 670.418606] Lustre: Skipped 2 previous similar messages [ 675.510157] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 685.501301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 689.913524] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 692.435891] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 698.521474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 706.031651] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 714.575746] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 714.760436] Lustre: lustre-OST0001: new disk, initializing [ 714.763935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 714.774168] Lustre: 26021:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 715.969950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 715.992502] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 716.120840] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 721.272898] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 730.803101] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 734.437504] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 746.979913] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 746.993126] Lustre: Skipped 3 previous similar messages [ 746.994374] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 747.009019] Lustre: Skipped 5 previous similar messages [ 753.490538] Lustre: server umount lustre-MDT0000 complete [ 757.223182] LustreError: 23697:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 757.245793] LustreError: 23697:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 757.370515] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614330 with bad export cookie 7527021833172386485 [ 757.381358] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 757.384255] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 757.664108] Lustre: server umount lustre-MDT0001 complete [ 771.617331] Lustre: server umount lustre-OST0000 complete [ 785.288556] Lustre: server umount lustre-OST0001 complete [ 802.826891] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 813.202760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 813.643536] LustreError: 29043:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 813.737945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 813.755943] Lustre: Skipped 3 previous similar messages [ 818.805874] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 827.704850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.158765] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 836.265489] Lustre: 30191:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 844.028326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 852.978820] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 861.783672] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 869.178032] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 877.683967] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 881.399052] Lustre: 32070:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 889.321780] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 889.334936] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 889.352754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 889.363331] Lustre: Skipped 4 previous similar messages [ 894.262292] Lustre: server umount lustre-MDT0000 complete [ 898.042124] LustreError: 29040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 898.067894] LustreError: 29040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 898.173645] LustreError: 29025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614471 with bad export cookie 7527021833172387780 [ 898.175221] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 898.191179] LustreError: 29025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 898.540378] Lustre: server umount lustre-MDT0001 complete [ 911.492995] Lustre: server umount lustre-OST0000 complete [ 924.528146] Lustre: server umount lustre-OST0001 complete [ 933.404546] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:35:04 (1787614504) [ 962.153390] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 972.476805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 977.125818] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 985.481869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 990.202154] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 993.028752] Lustre: 36024:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 999.431714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1005.824165] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1013.759941] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1020.820384] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1027.858532] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1031.196882] Lustre: 37904:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1037.281187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1037.286652] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1037.308365] Lustre: Skipped 3 previous similar messages [ 1037.316243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1037.321392] Lustre: Skipped 4 previous similar messages [ 1042.319502] Lustre: server umount lustre-MDT0000 complete [ 1043.937506] LustreError: 34875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1043.962648] LustreError: 34875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 1046.191130] LustreError: 34856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614619 with bad export cookie 7527021833172388886 [ 1046.193285] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1046.201289] LustreError: 34856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1046.672428] Lustre: server umount lustre-MDT0001 complete [ 1061.101562] Lustre: server umount lustre-OST0000 complete [ 1073.483896] Lustre: server umount lustre-OST0001 complete [ 1081.668682] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:37:32 (1787614652) [ 1096.900476] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1107.167458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1107.654855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1107.669251] Lustre: Skipped 7 previous similar messages [ 1112.304890] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1119.823710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1123.507890] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1125.916119] Lustre: 41198:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1132.461912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1139.621381] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1147.889806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1152.805191] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1159.664659] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1162.818947] Lustre: 43074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1163.510684] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1163.521364] Lustre: Skipped 1 previous similar message [ 1166.306436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1166.316503] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1166.326530] Lustre: Skipped 3 previous similar messages [ 1172.105990] Lustre: server umount lustre-MDT0000 complete [ 1175.156477] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614748 with bad export cookie 7527021833172389992 [ 1175.168666] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1175.172221] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1181.465157] Lustre: server umount lustre-MDT0001 complete [ 1188.764364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1204.799500] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1211.373393] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1212.211069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1218.080858] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1218.090890] Lustre: Skipped 1 previous similar message [ 1218.434360] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1225.783032] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1227.419369] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1233.112825] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1234.887600] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1243.629066] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1246.720730] Lustre: server umount lustre-MDT0000 complete [ 1250.419845] LustreError: 41577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614823 with bad export cookie 7527021833172391119 [ 1250.430821] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1250.436413] LustreError: 41577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1250.869426] Lustre: server umount lustre-MDT0001 complete [ 1265.568137] Lustre: server umount lustre-OST0000 complete [ 1279.564102] Lustre: server umount lustre-OST0001 complete [ 1288.268479] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:40:59 (1787614859) [ 1303.700718] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1313.279876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1313.684345] LustreError: 47681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1313.703640] LustreError: 47681:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 32 previous similar messages [ 1318.327674] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1326.348832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1330.463808] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1332.951411] Lustre: 48836:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1339.288975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1345.108342] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1353.229446] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1360.317247] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1368.724732] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1373.191415] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1388.005729] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1388.011859] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1388.016168] Lustre: Skipped 20 previous similar messages [ 1391.962379] Lustre: server umount lustre-MDT0000 complete [ 1395.568893] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787614968 with bad export cookie 7527021833172391910 [ 1395.573889] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1395.583856] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1396.067258] Lustre: server umount lustre-MDT0001 complete [ 1410.212349] Lustre: server umount lustre-OST0000 complete [ 1423.896818] Lustre: server umount lustre-OST0001 complete [ 1433.277645] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:43:24 (1787615004) [ 1435.651859] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1437.410314] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:43:28 (1787615008) [ 1447.311254] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 1454.419600] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1474.367164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1474.610649] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1474.704458] Lustre: lustre-MDT0000: new disk, initializing [ 1474.773612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1478.704647] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1486.756825] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1493.404503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1493.934361] Lustre: lustre-OST0000: new disk, initializing [ 1493.937704] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1493.944616] Lustre: Skipped 1 previous similar message [ 1493.951945] Lustre: 54908:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1495.472507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1495.480068] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1495.506794] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1499.989266] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1510.673203] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1512.871790] Lustre: DEBUG MARKER: create test files [ 1546.723492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1546.728187] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1546.734151] Lustre: Skipped 13 previous similar messages [ 1549.002391] Lustre: server umount lustre-OST0000 complete [ 1552.865562] Lustre: server umount lustre-MDT0000 complete [ 1563.048788] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1565.323705] Key type lgssc unregistered [ 1565.606517] LNet: 56539:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1565.611618] LNetError: 56539:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1565.626801] LNet: Removed LNI 192.168.203.116@tcp [ 1566.610868] Key type .llcrypt unregistered [ 1566.613124] Key type ._llcrypt unregistered [ 1592.320183] Key type ._llcrypt registered [ 1592.322202] Key type .llcrypt registered [ 1592.469669] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1593.188093] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1593.214723] alg: No test for adler32 (adler32-zlib) [ 1594.280599] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 1594.514268] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1596.168152] Key type lgssc registered [ 1597.348041] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1604.358354] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1604.378594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1605.813284] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1610.320740] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1617.389959] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1623.362643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1623.634031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1629.580035] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1630.843343] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1630.899607] Lustre: 59253:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1630.915628] Lustre: 59253:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1630.921576] Lustre: 59253:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1630.926741] Lustre: 59253:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1630.933570] Lustre: 59253:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1630.940173] Lustre: 59253:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1637.695432] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1642.622336] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1647.919600] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1655.573424] Lustre: server umount lustre-OST0000 complete [ 1669.268909] Lustre: server umount lustre-MDT0000 complete [ 1678.820310] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1680.912977] Key type lgssc unregistered [ 1681.173811] LNet: 60867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1681.180272] LNetError: 60867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1681.191913] LNet: Removed LNI 192.168.203.116@tcp [ 1681.895918] Key type .llcrypt unregistered [ 1681.898303] Key type ._llcrypt unregistered [ 1706.794161] Key type ._llcrypt registered [ 1706.796451] Key type .llcrypt registered [ 1706.876389] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1707.603646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1707.625163] alg: No test for adler32 (adler32-zlib) [ 1708.598133] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 1708.807876] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1710.472810] Key type lgssc registered [ 1711.421852] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1717.400322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1717.416367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1718.682988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1721.894969] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1727.548807] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1732.802453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1733.073873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1738.155573] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1743.341499] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1743.411061] Lustre: 63695:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1743.418427] Lustre: 63695:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1743.425801] Lustre: 63695:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1743.431802] Lustre: 63695:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1743.438315] Lustre: 63695:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1743.444622] Lustre: 63695:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1743.822775] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1749.359200] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1758.692949] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1758.700886] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1758.716486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1762.149742] Lustre: server umount lustre-OST0000 complete [ 1766.049897] Lustre: server umount lustre-MDT0000 complete [ 1776.040742] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1778.445517] Key type lgssc unregistered [ 1778.764617] LNet: 65256:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1778.768334] LNetError: 65256:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1778.794299] LNet: Removed LNI 192.168.203.116@tcp [ 1779.631865] Key type .llcrypt unregistered [ 1779.636233] Key type ._llcrypt unregistered [ 1796.749736] Key type ._llcrypt registered [ 1796.753307] Key type .llcrypt registered [ 1796.899266] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 1815.238226] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1816.673474] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1816.721912] alg: No test for adler32 (adler32-zlib) [ 1817.705348] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 1817.943455] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1819.584203] Key type lgssc registered [ 1820.583909] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1851.612169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1851.651119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1852.862452] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1852.883833] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1852.953266] Lustre: lustre-MDT0000: new disk, initializing [ 1853.029977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1853.049292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1856.879258] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1867.371194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1867.472462] Lustre: 68409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1867.552271] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1867.565137] Lustre: Skipped 1 previous similar message [ 1867.646766] Lustre: lustre-MDT0001: new disk, initializing [ 1867.811431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1867.880488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1867.906381] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1873.927253] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1882.452721] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1887.663426] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1893.817144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1894.036314] Lustre: lustre-OST0000: new disk, initializing [ 1894.039956] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1894.045528] Lustre: 70378:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1894.149295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1899.317239] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1901.119593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1901.129383] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1901.253683] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1908.437119] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1913.285725] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1913.452965] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1917.754341] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1917.976830] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1922.020677] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1922.024963] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1922.032702] LustreError: Skipped 1 previous similar message [ 1922.056728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1922.064850] Lustre: Skipped 1 previous similar message [ 1925.792500] Lustre: server umount lustre-OST0000 complete [ 1929.187611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1929.199200] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1929.213707] Lustre: Skipped 1 previous similar message [ 1929.221320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1932.259355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1932.260669] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1935.265829] Lustre: server umount lustre-MDT0000 complete [ 1938.276935] LustreError: 68400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787615511 with bad export cookie 5783337790196922616 [ 1938.278478] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1938.286770] LustreError: 68400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1938.667585] Lustre: server umount lustre-MDT0001 complete [ 1945.345869] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:51:56 (1787615516) [ 1955.323676] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 1990.480885] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 2000.887819] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2040.462249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2040.624317] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2040.664568] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2040.732978] Lustre: lustre-MDT0000: new disk, initializing [ 2040.794493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2040.820420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2045.165728] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2057.266695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2057.389728] Lustre: 76398:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2057.478122] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2057.480592] Lustre: Skipped 1 previous similar message [ 2057.535338] Lustre: lustre-MDT0001: new disk, initializing [ 2057.665187] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2057.691848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2057.703085] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2061.528504] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2070.641048] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2075.909618] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2081.677627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2081.929752] Lustre: lustre-OST0000: new disk, initializing [ 2081.932872] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2081.940566] Lustre: 78366:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2082.020795] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2088.316155] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2089.037539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2089.059307] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2089.157048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2097.383875] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2101.347588] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2101.606714] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2105.598423] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2105.829928] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2109.940013] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2109.943813] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2109.946618] LustreError: Skipped 1 previous similar message [ 2109.959728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2109.964547] Lustre: Skipped 1 previous similar message [ 2109.967626] Lustre: Skipped 1 previous similar message [ 2114.031972] Lustre: server umount lustre-OST0000 complete [ 2119.137581] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2119.150783] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2119.160851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2119.168917] Lustre: Skipped 1 previous similar message [ 2124.152037] Lustre: server umount lustre-MDT0000 complete [ 2125.297173] LustreError: 77624:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2125.326433] LustreError: 77624:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2127.821057] LustreError: 76389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787615700 with bad export cookie 5783337790196923498 [ 2127.830413] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2127.831424] LustreError: 76389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2128.096965] Lustre: server umount lustre-MDT0001 complete [ 2137.093526] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:55:07 (1787615707) [ 2143.348256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2143.667309] LustreError: 80778:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2143.729766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2148.354562] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2148.836029] LustreError: 80779:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2153.963246] LustreError: 80778:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2156.018102] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2161.135556] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2168.229831] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2172.528397] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2178.153829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2178.475306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2178.484292] Lustre: Skipped 1 previous similar message [ 2183.562769] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2190.725910] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2194.543368] Lustre: 83448:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 1532de52-0dd9-47fb-bc9c-18c4c341fa65 at adminstrative request [ 2199.032555] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2199.055608] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2208.247410] Lustre: MGS: Client 1532de52-0dd9-47fb-bc9c-18c4c341fa65 (at 0@lo) reconnecting [ 2208.256522] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x50428b1428c7bfd5 to 0x50428b1428c7c27c [ 2208.286614] Lustre: MGC192.168.203.116@tcp: Connection restored to 0@lo (at 0@lo) [ 2232.830599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2233.076736] Lustre: lustre-OST0001: new disk, initializing [ 2233.085300] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2233.089905] Lustre: 83842:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2239.598647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2239.608050] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2239.725767] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2240.377082] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2250.471594] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2255.334884] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2255.341881] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2255.351412] Lustre: Skipped 2 previous similar messages [ 2255.360828] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2255.366361] Lustre: Skipped 3 previous similar messages [ 2259.568948] Lustre: server umount lustre-OST0001 complete [ 2260.456223] LustreError: 82601:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2260.477231] LustreError: 82601:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2263.523939] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2263.527014] LustreError: Skipped 1 previous similar message [ 2268.751377] Lustre: server umount lustre-OST0000 complete [ 2274.278338] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2274.295888] Lustre: Skipped 2 previous similar messages [ 2274.302181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2274.309746] Lustre: Skipped 3 previous similar messages [ 2279.695699] Lustre: server umount lustre-MDT0000 complete [ 2280.929830] LustreError: 81782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2280.967168] LustreError: 81782:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2284.297837] LustreError: 80758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787615857 with bad export cookie 5783337790196925052 [ 2284.302256] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2284.310296] LustreError: 80758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2284.582985] Lustre: server umount lustre-MDT0001 complete [ 2293.732822] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:57:44 (1787615864) [ 2295.480816] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2297.341440] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:57:48 (1787615868) [ 2299.080456] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2300.925870] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:57:51 (1787615871) [ 2302.759807] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2304.634487] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 19:57:55 (1787615875) [ 2306.073174] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2307.734379] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:57:58 (1787615878) [ 2309.068226] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2313.987855] loop: module loaded [ 2313.998364] loop0: detected capacity change from 0 to 4294967296 [ 2314.041246] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2315.413606] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2318.483987] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2322.068769] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2346.608549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2346.654427] LustreError: 87965:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2346.664208] LustreError: 87965:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2346.670217] LustreError: 87965:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2346.686440] LustreError: 87965:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2369.029588] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 2378.849274] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2420.051237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2420.319117] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2420.345971] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2420.436391] Lustre: lustre-MDT0000: new disk, initializing [ 2420.553953] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2420.568595] Lustre: Skipped 1 previous similar message [ 2420.616417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2424.665693] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2434.166847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2434.287390] Lustre: 90831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2434.586158] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2438.559496] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2448.869400] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2455.406356] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2462.929197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2463.234385] Lustre: 92798:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2468.647505] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2471.559638] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2471.698339] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2477.763835] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2482.138551] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2482.326909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2487.125908] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2487.350432] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2492.388225] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2492.390117] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2492.393027] LustreError: Skipped 2 previous similar messages [ 2492.399933] Lustre: Skipped 1 previous similar message [ 2492.401353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2492.412541] Lustre: Skipped 3 previous similar messages [ 2495.685451] Lustre: server umount lustre-OST0000 complete [ 2505.836111] Lustre: server umount lustre-MDT0000 complete [ 2507.746546] LustreError: 90838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2507.769047] LustreError: 90838:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2509.321230] LustreError: 90824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787616082 with bad export cookie 5783337790196925668 [ 2509.323352] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2509.335103] LustreError: 90824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2510.061300] Lustre: server umount lustre-MDT0001 complete [ 2519.287247] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:01:30 (1787616090) [ 2525.807717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2531.194116] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2541.080327] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2546.363723] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2554.221789] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2559.910394] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2566.817849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2567.201784] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2567.211521] Lustre: Skipped 4 previous similar messages [ 2573.635206] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2581.067549] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2585.793493] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2586.075250] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2590.851800] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2591.084343] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2598.088578] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2598.316507] Lustre: lustre-OST0001: new disk, initializing [ 2598.320715] Lustre: Skipped 2 previous similar messages [ 2598.325379] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2598.331129] Lustre: Skipped 3 previous similar messages [ 2598.339971] Lustre: 98594:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2604.431924] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2607.172359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2607.190857] Lustre: Skipped 2 previous similar messages [ 2607.203043] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2607.339448] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2614.395941] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2619.647410] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2619.869604] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2624.178825] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2624.371856] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2643.520153] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2670.088665] Lustre: Failing over lustre-MDT0000 [ 2670.519448] Lustre: server umount lustre-MDT0000 complete [ 2672.478898] LustreError: 95220:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2672.494494] LustreError: 95220:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2674.145141] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2674.166253] Lustre: Skipped 6 previous similar messages [ 2689.507582] Lustre: 66635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787616246/real 1787616247] req@00000000deffddd7 x1874450595450112/t0(0) o400->MGC192.168.203.116@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787616262 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2689.544888] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2694.211443] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2694.213986] LDISKFS-fs (dm-0): recovery complete [ 2694.219568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2699.754387] LustreError: 66631:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000b84f59db x1874450595458176/t0(0) o250->MGC192.168.203.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2700.120673] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2703.182585] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2703.194480] Lustre: 95219:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2704.207736] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2705.383119] Lustre: 95223:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2705.393740] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2705.403907] Lustre: 95223:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2705.494222] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2705.818552] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2706.157705] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2706.166170] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2706.488131] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2706.497203] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2706.514219] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2706.527597] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2707.169667] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2707.188167] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2707.521791] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2707.528283] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2708.834194] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2708.856224] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2709.177145] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2709.187639] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2709.521274] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2709.527203] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2713.017195] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2713.037464] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2713.366059] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2713.389915] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2713.712935] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2713.728265] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2721.315797] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2721.329896] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2721.653722] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2721.663170] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2721.984726] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2721.993371] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2737.483038] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2737.505574] Lustre: 101445:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2737.825080] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2737.829145] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2738.144828] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2738.153243] LustreError: 101445:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2763.312163] Lustre: 101445:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2763.326239] Lustre: 101445:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2763.339092] Lustre: 101445:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 7c2cbc60-d51f-46ea-a7cc-169356edd204@192.168.203.16@tcp [ 2763.349339] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2763.362441] Lustre: 101445:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000f8483414 x1874450576381440/t0(8589934778) o35->7c2cbc60-d51f-46ea-a7cc-169356edd204@192.168.203.16@tcp:581/0 lens 392/0 e 0 to 0 dl 1787616346 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2763.409747] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2763.428836] Lustre: lustre-MDT0000: Denying connection for new client 7c2cbc60-d51f-46ea-a7cc-169356edd204 (at 192.168.203.16@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2763.435812] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2763.460599] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2763.463724] Lustre: Skipped 2 previous similar messages [ 2763.468454] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2763.511802] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2763.516612] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2776.041147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2776.051834] LustreError: Skipped 2 previous similar messages [ 2776.070359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2776.084725] Lustre: Skipped 3 previous similar messages [ 2790.368140] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2790.468312] Lustre: server umount lustre-OST0000 complete [ 2792.421484] LustreError: 101824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2792.446094] LustreError: 101824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 36 previous similar messages [ 2800.511536] Lustre: server umount lustre-OST0001 complete [ 2804.706530] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2804.724854] Lustre: Skipped 4 previous similar messages [ 2810.680779] Lustre: server umount lustre-MDT0000 complete [ 2814.216244] LustreError: 98592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787616387 with bad export cookie 5783337790196976887 [ 2814.227617] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2814.231222] LustreError: 98592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2814.611792] Lustre: server umount lustre-MDT0001 complete [ 2823.207744] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:06:34 (1787616394) [ 2831.351415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2831.356672] Lustre: *** cfs_fail_loc=197, val=0*** [ 2847.010541] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2852.132142] LustreError: 103422:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 2857.274584] Lustre: *** cfs_fail_loc=197, val=0*** [ 2857.278712] Lustre: Skipped 65 previous similar messages [ 2877.856987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2877.865090] Lustre: Skipped 2 previous similar messages [ 2884.347443] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2891.832903] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2893.867481] Lustre: server umount lustre-OST0000 complete [ 2901.479062] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:07:52 (1787616472) [ 2921.147141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2921.598775] LustreError: 105042:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2921.614731] LustreError: 105042:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2926.043822] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2935.531554] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2940.058981] LustreError: 105073:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2940.079855] LustreError: 105073:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2940.100935] LustreError: 105073:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 2940.458534] Lustre: server umount lustre-MDT0000 complete [ 2975.027376] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 2984.785252] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3020.381629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3020.577537] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3020.600621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3020.681484] Lustre: lustre-MDT0000: new disk, initializing [ 3020.753375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3024.759929] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3036.165198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3036.317488] Lustre: 109245:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3036.600455] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3041.592138] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3051.597284] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3057.200502] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3063.969962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3064.223489] Lustre: 111215:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3070.051885] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3071.681710] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3079.635493] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3084.789608] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3085.008824] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3089.903638] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3090.134774] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3092.450872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3092.457409] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3092.473328] Lustre: Skipped 2 previous similar messages [ 3092.473491] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3092.473494] Lustre: Skipped 12 previous similar messages [ 3092.502457] LustreError: Skipped 4 previous similar messages [ 3106.272683] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3106.488313] Lustre: server umount lustre-OST0000 complete [ 3116.312640] Lustre: server umount lustre-MDT0000 complete [ 3119.907606] LustreError: 109239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787616692 with bad export cookie 5783337790196987562 [ 3119.921810] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3120.246605] Lustre: server umount lustre-MDT0001 complete [ 3129.131805] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:11:40 (1787616700) [ 3137.293418] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3140.490605] Key type lgssc unregistered [ 3140.919776] LNet: 113708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3140.936319] LNetError: 113708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3141.992248] LNet: Removed LNI 192.168.203.116@tcp [ 3142.962765] Key type .llcrypt unregistered [ 3142.965299] Key type ._llcrypt unregistered [ 3162.884107] Key type ._llcrypt registered [ 3162.887103] Key type .llcrypt registered [ 3163.019583] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3178.832775] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3180.034223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3180.078866] alg: No test for adler32 (adler32-zlib) [ 3181.322832] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 3181.595577] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3183.280239] Key type lgssc registered [ 3184.521726] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3225.748903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3225.770335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3227.086399] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3227.116757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3227.194310] Lustre: lustre-MDT0000: new disk, initializing [ 3227.266964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3227.283783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3231.637421] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3241.873703] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3242.223352] Lustre: lustre-OST0000: new disk, initializing [ 3242.226566] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3242.235837] Lustre: Skipped 1 previous similar message [ 3242.244258] Lustre: 117882:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3242.385381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3243.291424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3243.316240] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3243.361740] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3248.623741] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3258.975564] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3259.174894] Lustre: lustre-OST0001: new disk, initializing [ 3259.178737] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3259.187628] Lustre: 118747:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3259.278721] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3265.053449] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3269.227382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3269.245944] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3269.279376] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3274.219242] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3274.221323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3274.235378] Lustre: Skipped 1 previous similar message [ 3274.249798] Lustre: Skipped 1 previous similar message [ 3279.182312] Lustre: server umount lustre-MDT0000 complete [ 3285.802326] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3290.596121] LustreError: 118746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787616863 with bad export cookie 6259221209481873978 [ 3290.603498] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3290.614545] LustreError: 118746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3290.786665] Lustre: server umount lustre-OST0000 complete [ 3294.623818] Lustre: server umount lustre-OST0001 complete [ 3303.334202] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 20:14:34 (1787616874) [ 3325.038366] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 20:14:56 (1787616896) [ 3336.277577] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3360.339246] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:15:30 (1787616930) [ 3400.353400] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3415.569214] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3454.202465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3454.433270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3454.457894] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3454.563626] Lustre: lustre-MDT0000: new disk, initializing [ 3454.690896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3454.708610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3459.417286] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3470.253388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3470.410429] Lustre: 126523:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3470.473109] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3470.476966] Lustre: Skipped 1 previous similar message [ 3470.551796] Lustre: lustre-MDT0001: new disk, initializing [ 3470.656876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3470.694344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3470.708571] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3475.324482] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3485.030908] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3491.530525] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3498.709451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3498.987698] Lustre: lustre-OST0000: new disk, initializing [ 3498.994624] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3499.014535] Lustre: 128491:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3499.191545] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3506.762907] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3507.349649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3507.374419] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3507.493614] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3517.181355] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3522.002631] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3522.159316] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3526.991635] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3527.293157] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3533.282276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3533.292820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3533.304487] Lustre: Skipped 1 previous similar message [ 3535.584059] Lustre: server umount lustre-OST0000 complete [ 3542.497712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3542.513456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3542.539805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3542.542602] Lustre: Skipped 1 previous similar message [ 3545.581085] Lustre: server umount lustre-MDT0000 complete [ 3549.720317] LustreError: 126514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787617122 with bad export cookie 6259221209481875196 [ 3549.742576] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3550.085868] Lustre: server umount lustre-MDT0001 complete [ 3558.377073] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:18:49 (1787617129) [ 3565.662867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3566.106356] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3566.140330] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3566.185210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3570.583236] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3571.681727] LustreError: 130906:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3575.805304] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.629436] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3584.941447] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3593.357976] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3598.884360] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3604.504594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3604.717343] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3604.725278] Lustre: Skipped 1 previous similar message [ 3610.385555] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3619.908415] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3627.865404] Lustre: *** cfs_fail_loc=159, val=0*** [ 3644.332795] Lustre: lustre-MDT0000: Client 7a61dbf8-121f-4e39-94c2-e01d9ba6ad43 (at 192.168.203.16@tcp) reconnecting [ 3646.230528] Lustre: *** cfs_fail_loc=159, val=0*** [ 3646.236901] Lustre: Skipped 5 previous similar messages [ 3661.828996] Lustre: lustre-MDT0000: Client 7a61dbf8-121f-4e39-94c2-e01d9ba6ad43 (at 192.168.203.16@tcp) reconnecting [ 3667.434029] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3667.445756] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3667.459147] Lustre: Skipped 1 previous similar message [ 3667.471561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3667.476569] Lustre: Skipped 2 previous similar messages [ 3668.977746] Lustre: server umount lustre-OST0000 complete [ 3675.630579] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3675.639496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3675.652858] Lustre: Skipped 1 previous similar message [ 3675.682398] Lustre: Skipped 1 previous similar message [ 3679.483133] Lustre: server umount lustre-MDT0000 complete [ 3680.755356] LustreError: 130900:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3680.774743] LustreError: 130900:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3683.438340] LustreError: 130885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787617256 with bad export cookie 6259221209481876078 [ 3683.443237] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3683.462897] LustreError: 130885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3694.267180] Lustre: server umount lustre-MDT0001 complete [ 3707.669990] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3710.626364] Key type lgssc unregistered [ 3710.946084] LNet: 134681:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3710.958434] LNetError: 134681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3710.989682] LNet: Removed LNI 192.168.203.116@tcp [ 3712.057269] Key type .llcrypt unregistered [ 3712.059236] Key type ._llcrypt unregistered [ 3738.574275] Key type ._llcrypt registered [ 3738.576879] Key type .llcrypt registered [ 3738.695951] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3739.654188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3739.708491] alg: No test for adler32 (adler32-zlib) [ 3740.710974] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 3741.047738] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3742.736216] Key type lgssc registered [ 3743.873695] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3752.239845] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3752.273121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3753.710311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3758.356546] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3763.172171] LustreError: 136099:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3763.207851] LustreError: 136099:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3767.058880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3767.549469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3772.777995] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3782.826362] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3793.007059] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3805.105133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3805.545084] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3812.641558] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3821.438385] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3824.003278] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3835.193294] Lustre: *** cfs_fail_loc=159, val=0*** [ 3850.616330] Lustre: lustre-MDT0000: Client ceb5bfb6-ad1a-412c-a0ee-93a0684fe773 (at 192.168.203.16@tcp) reconnecting [ 3853.230462] Lustre: *** cfs_fail_loc=159, val=0*** [ 3853.235052] Lustre: Skipped 1 previous similar message [ 3869.664674] Lustre: lustre-MDT0001: Client ceb5bfb6-ad1a-412c-a0ee-93a0684fe773 (at 192.168.203.16@tcp) reconnecting [ 3871.649675] Lustre: *** cfs_fail_loc=159, val=0*** [ 3871.663346] Lustre: Skipped 2 previous similar messages [ 3887.996778] Lustre: lustre-MDT0001: Client ceb5bfb6-ad1a-412c-a0ee-93a0684fe773 (at 192.168.203.16@tcp) reconnecting [ 3893.532636] Lustre: 139142:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3896.899099] Lustre: *** cfs_fail_loc=159, val=0*** [ 3896.911263] Lustre: Skipped 5 previous similar messages [ 3913.114059] Lustre: lustre-MDT0000: Client 68a09a25-ceae-4a67-8a37-169e9f20f823 (at 192.168.203.16@tcp) reconnecting [ 3914.751405] Lustre: *** cfs_fail_loc=159, val=0*** [ 3914.761216] Lustre: Skipped 15 previous similar messages [ 3931.040902] Lustre: lustre-MDT0000: Client 68a09a25-ceae-4a67-8a37-169e9f20f823 (at 192.168.203.16@tcp) reconnecting [ 3932.755371] Lustre: 139439:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3938.801805] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3938.805627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3938.811327] LustreError: Skipped 1 previous similar message [ 3938.846401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3941.859407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3941.862471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3941.897980] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3941.907839] LustreError: Skipped 3 previous similar messages [ 3944.736972] Lustre: server umount lustre-OST0000 complete [ 3951.585703] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3951.596083] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3951.622233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3951.633181] Lustre: Skipped 1 previous similar message [ 3954.939196] Lustre: server umount lustre-MDT0000 complete [ 3957.217598] LustreError: 136100:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3957.230880] LustreError: 136100:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3958.426790] LustreError: 137944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787617531 with bad export cookie 3293121469462078029 [ 3958.427508] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3958.441047] LustreError: 137944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3958.908986] Lustre: server umount lustre-MDT0001 complete [ 3970.507467] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3973.420166] Key type lgssc unregistered [ 3973.770777] LNet: 140493:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3973.780260] LNetError: 140493:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3973.799835] LNet: Removed LNI 192.168.203.116@tcp [ 3974.745843] Key type .llcrypt unregistered [ 3974.750238] Key type ._llcrypt unregistered [ 4001.848370] Key type ._llcrypt registered [ 4001.852765] Key type .llcrypt registered [ 4001.975958] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4003.215372] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4003.471440] alg: No test for adler32 (adler32-zlib) [ 4004.561327] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4004.761151] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4006.408140] Key type lgssc registered [ 4007.813963] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4016.040680] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4016.068616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4017.755652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4022.537111] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4027.884714] LustreError: 141908:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4027.898105] LustreError: 141908:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4031.427790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4031.787947] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4035.927571] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4043.118707] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4049.420278] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4056.720652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4057.075383] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4062.200595] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4063.849045] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4071.426807] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4076.544348] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4081.658400] Lustre: 144494:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4084.836884] Lustre: 144543:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4086.477073] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4087.408551] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4091.926067] Lustre: 144690:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4092.955176] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4093.987620] Lustre: 144798:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4102.121995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4102.122539] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4102.133407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4102.157714] Lustre: Skipped 1 previous similar message [ 4104.490818] Lustre: server umount lustre-OST0000 complete [ 4108.771826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4108.780610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4108.804430] Lustre: Skipped 1 previous similar message [ 4108.812055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4112.355615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4112.356829] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4114.905121] Lustre: server umount lustre-MDT0000 complete [ 4117.479828] LustreError: 142966:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4117.509413] LustreError: 142966:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4119.563442] LustreError: 141889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787617692 with bad export cookie 9546923924289994434 [ 4119.574629] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4119.582607] LustreError: 141889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4119.925519] Lustre: server umount lustre-MDT0001 complete [ 4131.041278] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4133.392581] Key type lgssc unregistered [ 4133.650038] LNet: 145850:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4133.662783] LNetError: 145850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4134.694743] LNet: Removed LNI 192.168.203.116@tcp [ 4135.549992] Key type .llcrypt unregistered [ 4135.552134] Key type ._llcrypt unregistered [ 4156.663783] Key type ._llcrypt registered [ 4156.666115] Key type .llcrypt registered [ 4156.763835] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4157.572550] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4157.656639] alg: No test for adler32 (adler32-zlib) [ 4158.729907] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4159.036441] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4160.737482] Key type lgssc registered [ 4161.712985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4168.423797] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4168.435385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4169.784156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4173.524960] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4174.817203] LustreError: 147265:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4174.826580] LustreError: 147265:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4179.947653] LustreError: 147266:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4181.614546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4181.970370] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4187.032610] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4194.719077] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4201.396360] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4208.101595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4208.778969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4214.969143] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4215.175976] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4223.029973] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4226.238650] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4232.917970] Lustre: *** cfs_fail_loc=159, val=0*** [ 4249.503164] Lustre: lustre-MDT0000: Client 0a92dc62-6991-4b75-a843-ce9e464800a7 (at 192.168.203.16@tcp) reconnecting [ 4255.210283] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4255.218969] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4255.241717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4257.099619] Lustre: server umount lustre-OST0000 complete [ 4262.378836] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4262.382511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4262.395029] Lustre: Skipped 1 previous similar message [ 4262.411908] Lustre: Skipped 1 previous similar message [ 4267.490327] LustreError: 147266:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4267.515336] LustreError: 147266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4268.940176] Lustre: server umount lustre-MDT0000 complete [ 4272.016061] LustreError: 147247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787617844 with bad export cookie 10944068783449346038 [ 4272.025978] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4272.031964] LustreError: 147247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4281.446914] Lustre: server umount lustre-MDT0001 complete [ 4292.691169] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4295.276724] Key type lgssc unregistered [ 4295.575967] LNet: 150934:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4295.589431] LNetError: 150934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4295.607982] LNet: Removed LNI 192.168.203.116@tcp [ 4296.442808] Key type .llcrypt unregistered [ 4296.446574] Key type ._llcrypt unregistered [ 4320.043665] Key type ._llcrypt registered [ 4320.046887] Key type .llcrypt registered [ 4320.135927] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4321.113583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4321.158560] alg: No test for adler32 (adler32-zlib) [ 4322.448581] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4322.791470] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4324.496793] Key type lgssc registered [ 4325.793291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4333.627525] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4333.639931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4335.099352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4339.521948] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4344.310117] LustreError: 152349:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4344.319222] LustreError: 152349:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4347.958870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4348.350643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4352.915197] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4360.549795] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4365.850565] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4372.469533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4372.922239] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4373.962314] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4379.347748] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4380.082354] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4386.789647] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4405.611297] Lustre: lustre-MDT0001: evicting c51ae9dd-2d97-4f8b-8350-bda973c1f750 (at 192.168.203.16@tcp) by administrative request [ 4405.619539] Lustre: Skipped 1 previous similar message [ 4435.440724] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4435.460611] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4435.493927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4436.455398] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4436.466270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4436.472300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4436.521154] Lustre: Skipped 1 previous similar message [ 4439.878456] Lustre: server umount lustre-OST0000 complete [ 4445.665626] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4445.683450] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4445.707682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4450.147761] Lustre: server umount lustre-MDT0000 complete [ 4451.819761] LustreError: 153080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4451.839830] LustreError: 153080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4454.112094] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787618026 with bad export cookie 15296494879129272839 [ 4454.115339] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4454.122067] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4454.451528] Lustre: server umount lustre-MDT0001 complete [ 4464.648287] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4467.100372] Key type lgssc unregistered [ 4467.390482] LNet: 156417:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4467.397182] LNetError: 156417:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4468.458144] LNet: Removed LNI 192.168.203.116@tcp [ 4469.272690] Key type .llcrypt unregistered [ 4469.277227] Key type ._llcrypt unregistered [ 4541.104597] Key type ._llcrypt registered [ 4541.106504] Key type .llcrypt registered [ 4541.204332] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4542.280590] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4542.312592] alg: No test for adler32 (adler32-zlib) [ 4543.414801] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4543.592390] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4545.256173] Key type lgssc registered [ 4546.331298] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4552.812491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4552.829909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4554.329471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4558.232829] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4564.455166] LustreError: 158908:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4564.469974] LustreError: 158908:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4565.457828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4565.756709] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4569.367495] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4575.896079] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4581.130763] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4586.985628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4587.277914] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4590.386166] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4592.252432] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4598.979285] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4599.610171] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4610.017540] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4610.027881] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4610.028608] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4612.722701] Lustre: server umount lustre-OST0000 complete [ 4617.188074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4617.197106] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4617.212130] Lustre: Skipped 1 previous similar message [ 4617.223656] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4617.236401] Lustre: Skipped 1 previous similar message [ 4620.257484] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4620.258316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4620.269545] Lustre: Skipped 1 previous similar message [ 4622.264941] Lustre: server umount lustre-MDT0000 complete [ 4625.379605] LustreError: 158909:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4625.389681] LustreError: 158909:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4625.691970] LustreError: 158890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787618198 with bad export cookie 6374697726525855381 [ 4625.699945] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4625.705059] LustreError: 158890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4625.998344] Lustre: server umount lustre-MDT0001 complete [ 4634.888490] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4638.003766] Key type lgssc unregistered [ 4638.373225] LNet: 162452:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4638.380316] LNetError: 162452:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4638.413110] LNet: Removed LNI 192.168.203.116@tcp [ 4639.415905] Key type .llcrypt unregistered [ 4639.418555] Key type ._llcrypt unregistered [ 4664.150922] Key type ._llcrypt registered [ 4664.155891] Key type .llcrypt registered [ 4664.271553] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4664.986286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4665.018901] alg: No test for adler32 (adler32-zlib) [ 4666.043890] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4666.277201] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4667.960200] Key type lgssc registered [ 4668.972297] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4682.824558] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:37:33 (1787618253) [ 4696.903304] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 4704.563545] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4732.470544] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4732.492617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4733.688374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4733.727126] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4733.791418] Lustre: lustre-MDT0000: new disk, initializing [ 4733.851241] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4733.867145] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4737.492276] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4750.011062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4750.263110] Lustre: 166718:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4750.354704] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4750.364068] Lustre: Skipped 1 previous similar message [ 4750.531872] Lustre: lustre-MDT0001: new disk, initializing [ 4750.719767] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4750.802416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4750.814104] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4756.973255] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4766.544677] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4770.693891] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4777.372744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4777.654702] Lustre: lustre-OST0000: new disk, initializing [ 4777.669637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4777.678289] Lustre: 168685:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4777.758903] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4783.508904] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4787.299467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4787.305706] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4787.438421] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4792.354596] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4808.164351] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4808.165334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4808.180611] LustreError: Skipped 1 previous similar message [ 4808.181325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4808.193733] Lustre: Skipped 1 previous similar message [ 4808.215215] Lustre: Skipped 1 previous similar message [ 4810.092142] Lustre: server umount lustre-OST0000 complete [ 4817.378179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4817.396125] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4817.420196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4818.408867] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4818.429432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4818.441444] Lustre: Skipped 1 previous similar message [ 4820.234959] Lustre: server umount lustre-MDT0000 complete [ 4824.386454] LustreError: 168684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787618397 with bad export cookie 12766497666678321411 [ 4824.424220] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4824.856096] Lustre: server umount lustre-MDT0001 complete [ 4836.728210] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4839.196849] Key type lgssc unregistered [ 4839.506904] LNet: 170594:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4839.512381] LNetError: 170594:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4839.536274] LNet: Removed LNI 192.168.203.116@tcp [ 4840.182407] Key type .llcrypt unregistered [ 4840.184390] Key type ._llcrypt unregistered [ 4870.224799] Key type ._llcrypt registered [ 4870.227912] Key type .llcrypt registered [ 4870.383302] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4871.638261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4871.648761] alg: No test for adler32 (adler32-zlib) [ 4872.878899] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 4873.179849] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4874.848460] Key type lgssc registered [ 4876.022840] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4882.362069] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4882.412268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4883.739552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4887.334312] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4893.171040] LustreError: 172013:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4893.196866] LustreError: 172013:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4894.241347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4894.562069] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4898.076906] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4904.341405] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4908.666501] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4913.581908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4913.831352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4918.257579] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4924.688602] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4928.178345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4935.426303] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4935.451973] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4935.488323] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4935.500213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.516902] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4936.136041] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4936.149911] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4936.167238] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4936.177760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.187505] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4936.929184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.662615] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4937.666465] Lustre: Skipped 1 previous similar message [ 4937.677940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4937.687335] Lustre: Skipped 1 previous similar message [ 4937.692522] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4937.699737] Lustre: Skipped 1 previous similar message [ 4937.705544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.718584] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4937.723353] Lustre: Skipped 1 previous similar message [ 4938.421401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.267277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.016803] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4940.021740] Lustre: Skipped 2 previous similar messages [ 4940.038791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4940.045425] Lustre: Skipped 2 previous similar messages [ 4940.048771] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4940.052415] Lustre: Skipped 2 previous similar messages [ 4940.055531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.064248] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4940.069627] Lustre: Skipped 2 previous similar messages [ 4940.787918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.403893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.996910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.713812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.549118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.472474] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4944.476646] Lustre: Skipped 5 previous similar messages [ 4944.492288] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4944.503532] Lustre: Skipped 5 previous similar messages [ 4944.508350] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4944.512857] Lustre: Skipped 5 previous similar messages [ 4944.517700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.531291] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4944.539226] Lustre: Skipped 5 previous similar messages [ 4945.243305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.958706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.612438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.210066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.874534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.674080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.350293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.196313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.973372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.843920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.798577] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4952.805604] Lustre: Skipped 10 previous similar messages [ 4952.818724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4952.828291] Lustre: Skipped 10 previous similar messages [ 4952.831846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4952.835059] Lustre: Skipped 10 previous similar messages [ 4952.838927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.852094] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4952.855046] Lustre: Skipped 10 previous similar messages [ 4953.799528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.744793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.563652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.230951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.979927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.668974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.460268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.149101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.915299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.818292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.824574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.667118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.545344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.440652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.247817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.072923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.061216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.989120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.903268] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4968.909741] Lustre: Skipped 18 previous similar messages [ 4968.928354] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4968.935832] Lustre: Skipped 18 previous similar messages [ 4968.939706] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4968.943874] Lustre: Skipped 18 previous similar messages [ 4968.947426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.958498] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4968.965401] Lustre: Skipped 18 previous similar messages [ 4969.705185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.468221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.312391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.103170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.933575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.765666] LustreError: 172033:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4973.774970] LustreError: 172033:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4973.777573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.462755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.428988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.106548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.829057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.509877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.249636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.845582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.559624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.431605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.148375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.135397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.983814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.952949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.794690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.578367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.484425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.492141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.486670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.332727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.106084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.848948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.673195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.858042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.016680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.967637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.149783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.261725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.309963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.234214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.300601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.162373] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5001.168524] Lustre: Skipped 36 previous similar messages [ 5001.178203] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5001.189184] Lustre: Skipped 36 previous similar messages [ 5001.193468] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5001.196281] Lustre: Skipped 36 previous similar messages [ 5001.200074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.209496] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5001.213904] Lustre: Skipped 36 previous similar messages [ 5002.233163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.290270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.038377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.743358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.406206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.054588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.734244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.310942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.939439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.554820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.217450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.826206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.531187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.299472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.986785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.575345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.230222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.118416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.810370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.474282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.216390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.006281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.973151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.009089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.960603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.816449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.575873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.285590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.258386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.912691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.923852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.880568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.815159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.591797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.489058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.240162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.987102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.770721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.596660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.355937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.129333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.874876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.736643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.542278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.454895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.303293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.087126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.878600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.746371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.482955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.331625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.113399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.940446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.870842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.781686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.561640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.415357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.038719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.994645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.156084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.032064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.839537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.577588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.497308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.374906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.359052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.149565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.014158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.861753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.866954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.757343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.662625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.334926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.263338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.056850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.019523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.966599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.925817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.107551] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5066.117950] Lustre: Skipped 78 previous similar messages [ 5066.140213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.152276] Lustre: Skipped 78 previous similar messages [ 5066.156241] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5066.159162] Lustre: Skipped 78 previous similar messages [ 5066.180723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.200118] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5066.204414] Lustre: Skipped 78 previous similar messages [ 5067.381753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.626122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.477550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.437270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.244895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.335349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.168981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.178136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.004542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.027584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.913336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.838731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.685161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.499639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.363753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.039974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.947702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.868469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.790023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.628913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.628810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.483962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.415119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.444186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.482587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.685879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.526179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.598399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.482527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.319888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.072890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.025560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.934108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.938506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.743496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.552440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.331587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.182833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.995285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.823258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.550819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.383483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.154420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.064170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.872084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.757742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.584509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.413885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.144690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.987214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.834690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.651438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.612239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.499063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.443610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.545090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.448854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.586403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.624358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.922414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.037471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.086805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.897833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.049401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.520904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.818468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.806204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.684509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.493165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.350375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.300609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.334951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.404903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.217222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.927172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.700625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.558512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.150947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.829325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.420535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.293201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.078257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.982275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.675757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.536805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.366575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.138518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.090480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.979987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.843304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.652722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.485767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.197541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.044352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.925850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.634166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.413533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.334157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.273548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.127559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.879654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.658833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.427737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.212245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.004939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.916759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.807831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.686982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.668342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.398651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.188326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.025536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.844964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.716069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.635923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.564560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.603049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.884147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.952659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.844553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.952075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.064095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.053561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.306441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.427511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.197922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.479322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.441425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.631390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.453253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.343980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.346348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.370852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.243973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.195671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.130859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.250624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.495424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.458114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.658799] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5194.666028] Lustre: Skipped 139 previous similar messages [ 5194.690277] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5194.698588] Lustre: Skipped 139 previous similar messages [ 5194.702170] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5194.706984] Lustre: Skipped 139 previous similar messages [ 5194.710713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.738986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5194.765111] Lustre: Skipped 139 previous similar messages [ 5196.123879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.574702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.397498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.325449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.096492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.246364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.533939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.523038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.371778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.477612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.256721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.206719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.267673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.340141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.322688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.154456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.953868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.837191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.966238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.955626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.953031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.883465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.761685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.565711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.580885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.271526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.059612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.718129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.447725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.287513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.156470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.821151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.663100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.413065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.307625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.159821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.083256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.977352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.853863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.558629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.463571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.244762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.168025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.888638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.758711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.586405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.528275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.381904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.258226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.080945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.922130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.844918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.760589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.554273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.457840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.379292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.298699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.083960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.997200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.777722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.595619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.584734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.540285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.376709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.431042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.394089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.449023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.535322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.688833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.930608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.005688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.068400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.919602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.176452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.339045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.426441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.572956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.753024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.952226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.060314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.910981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.966729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.069905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.028068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.206865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.318234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.250559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.548786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.039260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.163234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.191240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.446594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.476645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.699681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.708401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.694851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.756456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.661977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.136074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.596300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.926937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.825228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.887194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.748480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.472906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.340705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.244902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.064308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.943164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.050961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.157471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.130895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.455611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.676109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.656774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.602608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.269736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.991130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.964888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.856847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.041752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.034548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.781929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.436347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.303248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.001904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.729146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.693596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.507827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.331577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.608855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.921354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.644392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.465947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.594307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.489213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.578207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.602069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.431181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.316698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.365706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.329312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.241528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.123564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.111864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.003464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.998272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.807806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.764612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.635127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.582677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.872244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.828374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.843162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.984375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.922816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.680069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.569830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.621870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.601220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.641077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.646750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.472157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.313759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.345707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.156758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.080846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.747183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.787885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.006529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.842954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.741592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.513449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.680887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.599386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.436594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.648771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.532658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.415361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.306236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.216961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.049852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.969613] LustreError: 172033:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5370.981146] LustreError: 172033:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5370.996392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.660565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.483906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.299865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.346982] LustreError: 172033:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5374.366702] LustreError: 172033:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5374.376292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.260944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.207986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.984966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.105906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.063584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.947088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.023728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.114282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.200407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.320175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.387293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.738449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.953271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.981995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.276325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.962957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.885331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.842116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.826524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.807892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.062826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.070258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.050950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.911612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.828381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.887891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.911835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.159512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.244381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.206565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.520824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.712830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.715537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.711054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.074304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.036876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.998890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.270846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.869037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.098352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.221245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.526938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.803969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.953893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.974570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.921909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.979609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.834700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.945678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.971123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.124596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.329569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.271843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.312040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.051961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.164226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.890024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.828841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.772230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.465682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.184301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.126870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.236925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.592613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.641124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.612583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.596547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.627239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.711613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.738334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.830884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.802976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.691692] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5450.701200] Lustre: Skipped 259 previous similar messages [ 5450.716280] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5450.725377] Lustre: Skipped 259 previous similar messages [ 5450.730631] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5450.736863] Lustre: Skipped 259 previous similar messages [ 5450.741561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.753318] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5450.768420] Lustre: Skipped 259 previous similar messages [ 5451.757802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.946872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.791921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.802767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.630865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.374831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.228648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.253172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.161609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.143290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.022165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.007825] LustreError: 172033:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5462.020889] LustreError: 172033:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5462.037729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.943941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.947711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.844834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.845154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.715932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.922336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.790807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.706656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.801777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.728648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.665701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.442729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.350391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.555809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.404982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.242947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.274753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.248127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.050814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.141628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.093084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.168891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.120818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.164902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.153287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.267751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.260609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.232650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.076992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.091918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.990341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.958378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.984923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.362400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.663787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.968024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.116252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.742959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.783397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.087878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.627866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.401530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.519170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.767860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.951844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.975588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.332337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.542158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.370522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.551885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.692425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.737116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.596581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.757575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.074815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.295231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.571881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.110929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.183509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.687623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.114505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.541839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.949246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.395954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.407645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.444716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.346366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.725482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.841962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.089893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.115654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.033890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.349859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.566028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.737939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.336457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.255360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.926832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.968599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.236795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.320331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.280761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.381330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.198404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.968952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.946079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.762272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.800249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.653713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.688228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.455309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.317669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.203953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.068603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.870620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.720564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.702804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.530530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.883414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.262260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.186235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.070986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.959582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.814457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.669118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.423907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.276486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.045524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.818074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.695241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.610894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.460025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.263418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.231395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.230591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.033386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.915446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.756634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.755516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.793843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.876874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.674237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.687927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.606075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.392173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.261920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.230580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.004700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.821233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.466945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.351621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.533406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.325894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.363443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.422035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.693517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.739686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.829806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.100980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.410672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.743972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.192128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.445517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.774911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.706272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.626306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.833885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.847946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.683239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.454529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.269447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.457963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.059632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.817317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.207921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.191963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.197245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.364600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.186270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.325212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.631666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.731520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.842268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.775976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.643643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.365431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.229317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.103860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.982885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.794178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.738679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.941439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.979643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.041103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.928946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.979571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.744048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.718628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.609247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.801346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.128865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.325634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.571421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.426642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.541819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.657204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.669135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.459623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.083483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.829284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.688476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.708904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.646913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.896284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.760936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.665087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.608333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.865646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.528359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.358112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.296682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.074596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.804869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.549824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.297227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.059823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.674383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.496298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.378896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.387405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.153515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.090323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.150273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.035450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.117675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.957466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.829805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.900386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.846284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.725493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.369288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.103703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.003030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.958119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.729269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.447647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.283341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.209725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.204344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.983143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.746084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.642269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.516886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.318092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.257614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.191646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.416888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.596324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.649792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.664526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.543758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.581174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.397308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.655930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.700650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.979639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.213837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.357840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.365643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.458632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.238210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.055673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.865307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.622616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.425464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.299114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.165811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.997557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.903200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.013483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.849906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.678796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.546487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.576777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.407516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.387441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.107906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.907481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.812653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.882501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.907079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.824022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.570407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.476796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.125913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.784274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.531644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.491403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.261664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.019376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.775708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.699944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.741323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.979482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.825046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.608810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.469551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.406811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.346713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.157854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.969976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.964826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.836549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.693370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.672429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.553510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.524777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.335440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.544256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.816325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.064104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.325876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.493190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.377587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.322079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.214524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.054797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.021318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.042341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.212907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.215325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.350165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.262671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.020055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.181965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.369347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.266772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.214920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.366655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.441733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.389598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.270854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.535688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.492762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.884187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.234646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.635601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.942622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.872686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.633125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.596706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.540755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.436595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.371579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.517652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.823151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.799126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.769769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.649664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.612743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.712102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.725698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.673105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.576532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.276676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.964560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.563693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.171891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.924238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.553030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.320982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.897975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.458687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.153290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.026989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.874165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.817165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.808442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.785163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.727927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.638612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.714396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.617384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.544582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.431325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.370937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.316992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.405422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.547457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.599499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.560495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.553900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.491287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.327191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.228953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.066151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.942520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.851436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.722152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.477052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.183502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.122888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.045750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.883971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.747791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.546348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.396168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.161699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.043715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.785053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.497599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.182532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.846914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.702506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.505043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.260355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.096064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.977913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.844944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.738665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.549793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.287613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.068721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.800355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.660897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.335245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.232347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.987608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.700276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.433087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.159442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.818551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.708956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.446316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.172598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.853567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.545252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.184981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.881442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.520574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.286734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.007656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.744632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.526984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.263422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.969674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.801258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.581468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.323963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.662278] LustreError: 173832:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6294.677756] LustreError: 172033:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7097.317865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7097.321128] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7097.332536] Lustre: Skipped 442 previous similar messages [ 7097.341154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7102.449345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7102.464306] Lustre: Skipped 2 previous similar messages [ 7103.588324] Lustre: server umount lustre-OST0000 complete [ 7111.655814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7111.676711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7114.406302] Lustre: server umount lustre-MDT0000 complete [ 7117.796205] LustreError: 172013:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7117.811985] LustreError: 172013:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7118.637049] LustreError: 171995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787620691 with bad export cookie 13684355764617422551 [ 7118.638248] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7118.656767] LustreError: 171995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7119.329076] Lustre: server umount lustre-MDT0001 complete [ 7132.363808] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 7135.918290] Key type lgssc unregistered [ 7136.280079] LNet: 227316:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7136.287068] LNetError: 227316:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7136.303937] LNet: Removed LNI 192.168.203.116@tcp [ 7137.298206] Key type .llcrypt unregistered [ 7137.301156] Key type ._llcrypt unregistered [ 7166.278475] Key type ._llcrypt registered [ 7166.281795] Key type .llcrypt registered [ 7166.398924] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7167.457881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7167.551218] alg: No test for adler32 (adler32-zlib) [ 7168.677525] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 7168.975838] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 7170.696216] Key type lgssc registered [ 7172.199698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7180.324142] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7180.345966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7181.691949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7185.876370] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7192.041109] LustreError: 228732:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7192.073386] LustreError: 228732:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7195.103181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7195.500583] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7200.057503] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7207.027146] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7212.315795] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7218.836326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7219.151379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7225.925235] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7226.450143] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7235.891346] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7241.062823] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7243.455866] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7243.466361] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7243.472986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.488212] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7243.492780] LustreError: 228752:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7245.794883] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7245.802129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7245.812932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7247.863726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7247.868232] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7251.897245] Lustre: server umount lustre-OST0000 complete [ 7263.496484] LustreError: 228752:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7287.007800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7287.399062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7288.427983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.441540] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7293.995394] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7297.645637] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.670784] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7302.364245] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7304.353311] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7310.220871] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ef8d192d000.ost_server_uuid 50 [ 7311.799529] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ef8d192d000.ost_server_uuid in IDLE state after 0 sec [ 7317.474162] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7317.484758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7317.501348] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7317.508551] Lustre: Skipped 2 previous similar messages [ 7318.502183] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7322.231200] Lustre: server umount lustre-OST0000 complete [ 7328.737589] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7328.752825] Lustre: Skipped 1 previous similar message [ 7328.774248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7328.781286] Lustre: Skipped 3 previous similar messages [ 7332.650273] Lustre: server umount lustre-MDT0000 complete [ 7333.864032] LustreError: 228728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7333.877584] LustreError: 228728:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7336.004985] LustreError: 228711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787620908 with bad export cookie 2329410649411644609 [ 7336.006832] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7336.012476] LustreError: 228711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7336.453057] Lustre: server umount lustre-MDT0001 complete [ 7347.507851] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 7350.329481] Key type lgssc unregistered [ 7350.592515] LNet: 233940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7350.602694] LNetError: 233940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7350.632820] LNet: Removed LNI 192.168.203.116@tcp [ 7352.005360] Key type .llcrypt unregistered [ 7352.010321] Key type ._llcrypt unregistered [ 7392.904399] Key type ._llcrypt registered [ 7392.910920] Key type .llcrypt registered [ 7393.032723] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7394.161186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7394.243096] alg: No test for adler32 (adler32-zlib) [ 7395.385087] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 7395.662435] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 7397.328172] Key type lgssc registered [ 7398.468713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7409.135193] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7409.166545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7410.846398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7418.041880] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7420.403352] LustreError: 235670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7420.448851] LustreError: 235670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7425.516228] LustreError: 235671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7430.199569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7430.625535] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7430.916882] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7435.545514] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7438.597658] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7444.594743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7444.916409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7451.673102] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7456.185902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7459.676667] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7459.848262] Lustre: lustre-OST0001: new disk, initializing [ 7459.851909] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7459.857673] Lustre: 237983:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7459.962958] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7465.497446] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7470.205110] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7470.221230] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7470.355215] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7476.721528] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7485.077232] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7501.286918] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7501.298109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7501.299437] Lustre: Skipped 2 previous similar messages [ 7501.319379] Lustre: Skipped 3 previous similar messages [ 7506.408382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7506.422475] Lustre: Skipped 3 previous similar messages [ 7511.523554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7511.528940] Lustre: Skipped 3 previous similar messages [ 7514.080147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7514.279400] Lustre: server umount lustre-MDT0000 complete [ 7516.644434] LustreError: 235670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7516.670259] LustreError: 235670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7518.043588] LustreError: 235650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787621090 with bad export cookie 3844445063619301672 [ 7518.051160] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7518.057352] LustreError: 235650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7518.381028] Lustre: server umount lustre-MDT0001 complete [ 7532.429549] Lustre: server umount lustre-OST0000 complete [ 7547.349519] Lustre: server umount lustre-OST0001 complete [ 7568.685183] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7578.941586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7579.435422] LustreError: 241092:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7579.512499] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7584.223185] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7584.739163] LustreError: 241093:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7594.095290] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7594.586959] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7599.585080] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7602.458993] Lustre: 242232:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7608.853745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7614.853789] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7619.378799] LustreError: 242585:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7619.405131] LustreError: 242585:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7622.070758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7622.293478] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7622.300743] Lustre: Skipped 1 previous similar message [ 7624.389368] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7624.405128] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7627.767953] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7634.645263] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7637.714336] Lustre: 244105:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7650.298482] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7650.304679] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7650.321925] Lustre: Skipped 4 previous similar messages [ 7650.331605] Lustre: Skipped 2 previous similar messages [ 7660.516893] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7660.527401] Lustre: Skipped 7 previous similar messages [ 7662.052049] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7662.294266] Lustre: server umount mylustre-MDT0000 complete [ 7665.649586] LustreError: 244326:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7665.680894] LustreError: 244326:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7665.987718] LustreError: 241068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787621238 with bad export cookie 3844445063619304472 [ 7665.995462] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7666.000533] LustreError: 241068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7666.350598] Lustre: server umount mylustre-MDT0001 complete [ 7680.050202] Lustre: server umount mylustre-OST0000 complete [ 7694.776373] Lustre: server umount mylustre-OST0001 complete [ 7719.680786] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7730.675745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7731.080963] LustreError: 246418:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7731.170469] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7735.528981] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7744.531585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7749.925356] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7753.250140] Lustre: 247560:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7760.458238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7766.586565] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7774.911164] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7776.312892] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7776.329939] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7780.737834] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7788.306388] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7791.749407] Lustre: 249433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7801.316296] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7801.322958] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7801.337110] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7801.339526] Lustre: Skipped 1 previous similar message [ 7804.343241] Lustre: server umount tfs-MDT0000 complete [ 7806.950170] LustreError: 246414:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7806.971302] LustreError: 246414:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7807.701794] LustreError: 247562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787621380 with bad export cookie 3844445063619306369 [ 7807.703154] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7807.708686] LustreError: 247562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7808.009255] Lustre: server umount tfs-MDT0001 complete [ 7821.396715] Lustre: server umount tfs-OST0000 complete [ 7835.344449] Lustre: server umount tfs-OST0001 complete [ 7856.315790] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 7865.339474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7865.821936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7865.826360] Lustre: Skipped 3 previous similar messages [ 7869.647537] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7877.065605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7881.866711] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7884.631923] Lustre: 252823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7890.651808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7896.073744] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7897.093699] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7906.985765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7912.469118] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7916.667707] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 7927.081852] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7931.190175] Lustre: 254693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7939.823784] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:31:50 (1787621510) [ 7958.499385] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7958.518620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7958.552519] Lustre: Skipped 6 previous similar messages [ 7958.587336] Lustre: Skipped 6 previous similar messages [ 7963.620711] LustreError: 252415:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7963.630802] LustreError: 252415:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 7964.877114] Lustre: server umount lustre-MDT0000 complete [ 7971.042357] LustreError: 251657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787621543 with bad export cookie 3844445063619308035 [ 7971.058829] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7971.066197] LustreError: 251657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7973.869704] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7977.723783] Lustre: server umount lustre-MDT0001 complete [ 7997.408126] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7997.544927] Lustre: server umount lustre-OST0000 complete [ 8006.935404] Lustre: server umount lustre-OST0001 complete [ 8014.427226] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8027.334673] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8067.678287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8067.895846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8067.951450] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8068.053939] Lustre: lustre-MDT0000: new disk, initializing [ 8068.227505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8068.237080] Lustre: Skipped 3 previous similar messages [ 8068.265679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8072.938718] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8083.802286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8083.940943] Lustre: 257881:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8084.008170] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8084.012284] Lustre: Skipped 1 previous similar message [ 8084.110057] Lustre: lustre-MDT0001: new disk, initializing [ 8084.269876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8084.285764] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8088.861542] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8098.796169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8099.162714] Lustre: lustre-OST0000: new disk, initializing [ 8099.167971] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8099.178468] Lustre: 259466:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8100.507368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8100.523071] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8100.686274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8105.414347] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8117.353344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8117.625779] Lustre: lustre-OST0001: new disk, initializing [ 8117.630481] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8117.646440] Lustre: 260335:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8119.346455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8119.355305] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8119.492318] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8126.228697] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8141.750516] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:35:12 (1787621712) [ 8159.545466] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:35:30 (1787621730) [ 8177.642689] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8177.644789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8177.649669] LustreError: Skipped 1 previous similar message [ 8177.650831] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8177.664176] Lustre: Skipped 2 previous similar messages [ 8177.666884] Lustre: Skipped 4 previous similar messages [ 8182.390777] Lustre: server umount lustre-OST0000 complete [ 8186.857609] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8192.947261] Lustre: server umount lustre-MDT0000 complete [ 8196.440554] LustreError: 257873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787621769 with bad export cookie 3844445063619309589 [ 8196.449337] LustreError: 257873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8196.449869] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8196.584337] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8196.599854] Lustre: Skipped 3 previous similar messages [ 8202.796326] Lustre: server umount lustre-MDT0001 complete [ 8217.545081] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8292.456381] Lustre: server umount lustre-OST0001 complete [ 8300.552826] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8317.172318] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8356.624359] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8356.849441] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8356.880443] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8357.023417] Lustre: lustre-MDT0000: new disk, initializing [ 8357.145321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8357.154927] Lustre: Skipped 3 previous similar messages [ 8357.177202] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8361.529733] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8372.834932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8373.003687] Lustre: 265199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8373.059354] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8373.063421] Lustre: Skipped 1 previous similar message [ 8373.168678] Lustre: lustre-MDT0001: new disk, initializing [ 8373.298805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8373.309694] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8377.501342] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8386.705953] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8393.128807] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8400.853491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8401.234477] Lustre: 267166:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8402.604492] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8402.741231] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8408.072254] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8417.981821] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8428.513340] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8428.516923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8428.528324] LustreError: Skipped 1 previous similar message [ 8428.537259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8428.564055] Lustre: Skipped 1 previous similar message [ 8428.568353] Lustre: Skipped 9 previous similar messages [ 8431.523704] Lustre: server umount lustre-OST0000 complete [ 8441.855222] Lustre: server umount lustre-MDT0000 complete [ 8443.875331] LustreError: 266086:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8443.899745] LustreError: 266086:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 17 previous similar messages [ 8446.064559] LustreError: 265191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787622018 with bad export cookie 3844445063619311059 [ 8446.081107] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8446.084539] LustreError: 265191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8446.551576] Lustre: server umount lustre-MDT0001 complete [ 8458.895686] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8461.845365] Key type lgssc unregistered [ 8462.166773] LNet: 268927:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8462.173722] LNetError: 268927:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8462.201563] LNet: Removed LNI 192.168.203.116@tcp [ 8463.068149] Key type .llcrypt unregistered [ 8463.071117] Key type ._llcrypt unregistered [ 8490.861450] Key type ._llcrypt registered [ 8490.868160] Key type .llcrypt registered [ 8490.982300] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8492.037119] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8492.075027] alg: No test for adler32 (adler32-zlib) [ 8493.255196] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 8493.552547] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8495.266718] Key type lgssc registered [ 8496.484658] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8504.325134] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8504.343517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8505.741517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8510.230784] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8516.069409] LustreError: 270390:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8516.091841] LustreError: 270390:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8520.883896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8521.193636] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8521.515817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8527.212951] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8535.772206] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8542.252826] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8550.153141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8550.312612] LustreError: 272215:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8550.589111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8556.017538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8556.888634] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8565.343651] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8566.295340] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8571.364373] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8571.364582] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8571.375924] LustreError: Skipped 1 previous similar message [ 8571.393972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8571.404634] Lustre: Skipped 1 previous similar message [ 8574.402051] Lustre: server umount lustre-OST0000 complete [ 8578.019295] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8578.026896] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8578.042684] Lustre: Skipped 1 previous similar message [ 8578.051325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8581.612409] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8581.613795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8586.721613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8586.729805] Lustre: Skipped 2 previous similar messages [ 8592.352314] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8592.596231] Lustre: server umount lustre-MDT0000 complete [ 8596.963921] LustreError: 270390:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8596.988132] LustreError: 270390:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8597.022898] LustreError: 271125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787622169 with bad export cookie 17003700003971779527 [ 8597.023829] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8597.040362] LustreError: 271125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8597.598730] Lustre: server umount lustre-MDT0001 complete [ 8610.464510] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8614.350340] Key type lgssc unregistered [ 8614.676343] LNet: 273984:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8614.692490] LNetError: 273984:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8615.719824] LNet: Removed LNI 192.168.203.116@tcp [ 8616.700547] Key type .llcrypt unregistered [ 8616.709085] Key type ._llcrypt unregistered [ 8635.173943] Key type ._llcrypt registered [ 8635.175100] Key type .llcrypt registered [ 8635.261669] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8636.156145] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8636.181947] alg: No test for adler32 (adler32-zlib) [ 8637.267834] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 8637.542687] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8639.408927] Key type lgssc registered [ 8640.539958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8651.561521] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8651.606454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8667.168529] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8672.288397] LustreError: 275179:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 8698.912594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8707.870181] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 8716.711545] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8719.021375] Lustre: server umount lustre-OST0000 complete [ 8738.718830] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 8742.113265] Key type lgssc unregistered [ 8742.496900] LNet: 276814:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8742.508623] LNetError: 276814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8742.535235] LNet: Removed LNI 192.168.203.116@tcp [ 8743.653411] Key type .llcrypt unregistered [ 8743.658613] Key type ._llcrypt unregistered [ 8784.181512] Key type ._llcrypt registered [ 8784.188278] Key type .llcrypt registered [ 8784.278748] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 8785.382994] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8785.421082] alg: No test for adler32 (adler32-zlib) [ 8786.532847] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 8786.969894] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 8788.592703] Key type lgssc registered [ 8789.905577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8806.896905] loop0: detected capacity change from 0 to 204800000 [ 8807.178539] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8807.190361] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8807.419492] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8808.845885] loop0: detected capacity change from 0 to 204800000 [ 8809.060447] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8809.085463] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8809.671237] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8810.978947] loop0: detected capacity change from 0 to 204800000 [ 8811.184918] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8811.202385] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8811.350600] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8812.893803] loop0: detected capacity change from 0 to 204800000 [ 8813.141073] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8813.170178] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8813.321332] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8814.639816] loop0: detected capacity change from 0 to 204800000 [ 8814.712033] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8819.448480] loop0: detected capacity change from 0 to 204800000 [ 8819.474326] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8824.234776] loop0: detected capacity change from 0 to 204800000 [ 8824.274060] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8829.139251] loop0: detected capacity change from 0 to 204800000 [ 8829.194650] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8833.900957] loop0: detected capacity change from 0 to 204800000 [ 8834.052711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8834.114864] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8834.206809] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8834.219713] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8834.238751] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8834.254988] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8834.277805] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8834.295813] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8834.318399] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8834.332945] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8834.343389] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8834.358708] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8834.371553] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8834.382323] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8834.401704] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8834.419472] Lustre: 280237:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8840.306051] Lustre: server umount MGS complete [ 8841.356317] loop0: detected capacity change from 0 to 204800000 [ 8841.495956] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8842.046232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8842.133418] Lustre: lustre-MDT0000: Aborting client recovery [ 8842.987564] loop1: detected capacity change from 0 to 204800000 [ 8843.107348] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8843.230937] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8843.245386] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8843.275764] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8843.285815] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8843.293203] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8843.313136] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8843.330468] Lustre: 280671:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8843.372914] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8843.589037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8843.655097] Lustre: lustre-MDT0001: Aborting client recovery [ 8844.696384] loop2: detected capacity change from 0 to 204800000 [ 8844.870839] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8844.927784] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8844.934429] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8844.958979] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8844.977856] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8845.001755] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8845.017122] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8845.030412] Lustre: 280751:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8845.242795] Lustre: Found index 0 for lustre-OST0000, updating log [ 8845.320659] Lustre: 280769:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8845.411099] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8845.437403] Lustre: lustre-OST0000: Aborting recovery [ 8846.369462] loop3: detected capacity change from 0 to 204800000 [ 8846.540101] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8846.640478] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8846.670514] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8846.682872] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8846.691633] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8846.711662] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8846.731703] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8846.741284] Lustre: 280834:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8846.765708] Lustre: Found index 1 for lustre-OST0001, updating log [ 8846.807764] Lustre: 280839:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8846.925966] Lustre: lustre-OST0001: Aborting recovery [ 8851.967624] LustreError: 280615:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8852.965067] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8852.970056] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8853.064150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8853.121895] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8854.012118] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8854.013910] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8855.286379] LustreError: 280583:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 761us: rc = -2 [ 8855.816168] LustreError: 280583:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 849us: rc = -2 [ 8855.840624] LustreError: 280583:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8869.345748] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8869.348825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8869.352280] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8869.369590] Lustre: Skipped 2 previous similar messages [ 8869.383573] Lustre: Skipped 3 previous similar messages [ 8874.466460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8874.469352] Lustre: Skipped 3 previous similar messages [ 8877.509642] Lustre: server umount lustre-MDT0000 complete [ 8878.391146] LustreError: 280565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787622451 with bad export cookie 6606036189912030501 [ 8878.395600] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8878.401569] LustreError: 280565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8879.585089] LustreError: 280581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8879.587425] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8879.660570] LustreError: 280581:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8879.661956] Lustre: Skipped 1 previous similar message [ 8879.697819] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8879.732846] Lustre: Skipped 1 previous similar message [ 8883.617176] LustreError: 281070:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8883.618646] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8883.660143] Lustre: Skipped 1 previous similar message [ 8884.827160] Lustre: server umount lustre-MDT0001 complete [ 8892.136936] Lustre: server umount lustre-OST0000 complete [ 8899.688528] Lustre: server umount lustre-OST0001 complete [ 8908.185861] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:47:58 (1787622478) [ 8948.798867] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 8964.595352] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9010.667549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9011.049541] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9011.082715] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9011.160122] Lustre: lustre-MDT0000: new disk, initializing [ 9011.323946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9011.345968] Lustre: Skipped 1 previous similar message [ 9011.413309] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9016.523788] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9028.940323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9029.039454] Lustre: 284931:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9029.080810] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9029.095722] Lustre: Skipped 1 previous similar message [ 9029.190635] Lustre: lustre-MDT0001: new disk, initializing [ 9029.408979] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9029.496176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9029.520013] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9035.732157] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9048.893502] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9057.929777] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9067.681883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9068.031396] Lustre: lustre-OST0000: new disk, initializing [ 9068.042194] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9068.049617] Lustre: 286903:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9068.116413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9074.795966] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9078.347102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9078.359568] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9078.497724] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9084.379283] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9089.455921] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9098.564471] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9104.819547] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9113.213384] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9165.813140] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9165.836306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9165.860913] Lustre: Skipped 2 previous similar messages [ 9165.865590] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9165.942322] Lustre: Skipped 1 previous similar message [ 9176.041101] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9176.043718] Lustre: Skipped 2 previous similar messages [ 9178.080549] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9178.219377] Lustre: server umount lustre-OST0000 complete [ 9183.210329] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9183.218842] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9189.142106] Lustre: server umount lustre-MDT0000 complete [ 9191.396635] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9191.435568] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9192.836666] LustreError: 284924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787622765 with bad export cookie 6606036189912033210 [ 9192.840790] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9192.849627] LustreError: 284924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9193.425681] Lustre: server umount lustre-MDT0001 complete [ 9202.831768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9207.626536] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9212.137721] Lustre: server umount MGS complete [ 9224.702049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9225.395645] LustreError: 290727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9225.441722] LustreError: 290740:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9225.593209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9230.823037] LustreError: 290728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9231.253961] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9241.071504] LustreError: 290727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9241.098926] LustreError: 290727:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9243.914498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9250.212610] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9258.258132] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9263.551693] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9269.589567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9269.923626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9269.935424] Lustre: Skipped 1 previous similar message [ 9277.548524] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9288.106919] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9320.417777] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9320.430666] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9320.443184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9320.456991] Lustre: Skipped 2 previous similar messages [ 9320.472500] Lustre: Skipped 5 previous similar messages [ 9326.379584] Lustre: server umount lustre-OST0000 complete [ 9335.779046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9338.030534] Lustre: server umount lustre-MDT0000 complete [ 9340.897930] LustreError: 291457:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9340.953319] LustreError: 291457:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9344.133362] LustreError: 293455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787622916 with bad export cookie 6606036189912035135 [ 9344.164905] LustreError: 293455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9344.167688] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9344.598030] Lustre: server umount lustre-MDT0001 complete [ 9359.288585] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 9363.402950] Key type lgssc unregistered [ 9363.963656] LNet: 294678:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9363.975356] LNetError: 294678:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9364.009858] LNet: Removed LNI 192.168.203.116@tcp [ 9364.961109] Key type .llcrypt unregistered [ 9364.963221] Key type ._llcrypt unregistered [ 9407.488026] Key type ._llcrypt registered [ 9407.492895] Key type .llcrypt registered [ 9407.630747] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 9425.885800] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9427.167397] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9427.324913] alg: No test for adler32 (adler32-zlib) [ 9428.369937] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 9428.587967] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 9430.298573] Key type lgssc registered [ 9431.632629] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9471.183667] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9471.200819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9472.509092] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9472.554876] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9472.622753] Lustre: lustre-MDT0000: new disk, initializing [ 9472.717102] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9472.734340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9477.769752] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9489.516042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9489.668328] Lustre: 298645:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9489.719822] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9489.723467] Lustre: Skipped 1 previous similar message [ 9489.782440] Lustre: lustre-MDT0001: new disk, initializing [ 9489.871218] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9489.903721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9489.911095] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9495.284278] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9505.564769] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9512.034545] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9522.811849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9523.291557] Lustre: lustre-OST0000: new disk, initializing [ 9523.308625] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9523.335478] Lustre: 300617:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9523.493046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9531.014974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9531.026107] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9531.168922] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9532.220284] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9544.820695] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9553.432328] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9567.340925] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9576.588209] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9585.349369] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9628.647207] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9628.657630] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9628.670694] LustreError: Skipped 1 previous similar message [ 9628.678255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9628.689373] Lustre: Skipped 1 previous similar message [ 9628.723069] Lustre: Skipped 1 previous similar message [ 9631.821177] Lustre: server umount lustre-OST0000 complete [ 9638.369941] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9638.386404] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9638.404926] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9644.005804] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9644.015407] Lustre: Skipped 2 previous similar messages [ 9647.010436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9647.014745] Lustre: Skipped 1 previous similar message [ 9651.168154] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9651.397700] Lustre: server umount lustre-MDT0000 complete [ 9655.856658] LustreError: 302228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787623228 with bad export cookie 10572280897406632542 [ 9655.865758] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9656.125918] Lustre: server umount lustre-MDT0001 complete [ 9667.511922] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9673.953768] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9679.573891] Lustre: server umount MGS complete [ 9693.542966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9694.406879] LustreError: 304548:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9694.436828] LustreError: 304548:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9694.448724] LustreError: 304561:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9694.611577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9699.821180] LustreError: 304549:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9700.500622] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9705.100372] LustreError: 304548:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9710.569879] LustreError: 304549:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9713.850965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9714.740493] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9721.543925] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9732.342593] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9739.693994] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9747.241707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9747.666204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9755.132917] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9766.617370] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9801.193886] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9801.215467] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9801.245358] Lustre: Skipped 1 previous similar message [ 9801.250984] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9801.255043] Lustre: Skipped 2 previous similar messages [ 9807.106557] Lustre: server umount lustre-OST0000 complete [ 9811.938456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9811.948267] LustreError: Skipped 1 previous similar message [ 9811.951792] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9811.963561] Lustre: Skipped 1 previous similar message [ 9811.968903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9811.973767] Lustre: Skipped 2 previous similar messages [ 9816.736550] Lustre: server umount lustre-MDT0000 complete [ 9817.571328] LustreError: 304543:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9817.604661] LustreError: 304543:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9820.971743] LustreError: 304530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787623393 with bad export cookie 10572280897406634495 [ 9820.974329] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9820.991152] LustreError: 304530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9821.313656] Lustre: server umount lustre-MDT0001 complete [ 9834.078405] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 9836.950254] Key type lgssc unregistered [ 9837.212705] LNet: 308508:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9837.224419] LNetError: 308508:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9838.266045] LNet: Removed LNI 192.168.203.116@tcp [ 9839.463242] Key type .llcrypt unregistered [ 9839.465969] Key type ._llcrypt unregistered [ 9867.322674] Key type ._llcrypt registered [ 9867.324864] Key type .llcrypt registered [ 9867.394500] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 9882.655577] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9884.124583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9884.163306] alg: No test for adler32 (adler32-zlib) [ 9885.981320] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [ 9886.552748] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 9888.312429] Key type lgssc registered [ 9889.915903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9920.698704] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 9938.996930] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9939.021729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9940.230709] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9940.262260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9940.317493] Lustre: lustre-MDT0000: new disk, initializing [ 9940.360904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9940.372579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9945.771862] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9959.150611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9959.421620] Lustre: lustre-OST0000: new disk, initializing [ 9959.429372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9959.436089] Lustre: Skipped 1 previous similar message [ 9959.448146] Lustre: 312558:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9959.514233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9967.145886] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 9968.769485] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9968.784437] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9968.820532] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11109.556370] Lustre: server umount lustre-MDT0000 complete [11113.722263] LustreError: 311745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787624686 with bad export cookie 6917270748149410958 [11113.737854] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11123.407602] Lustre: server umount lustre-OST0000 complete [11151.831901] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [11167.598199] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11208.122136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11208.401385] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11208.446158] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11208.538184] Lustre: lustre-MDT0000: new disk, initializing [11208.628515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11208.653628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11213.780768] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11226.156962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11226.298963] Lustre: 316957:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11226.396884] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11226.407609] Lustre: Skipped 1 previous similar message [11226.512902] Lustre: lustre-MDT0001: new disk, initializing [11226.600502] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11226.650975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11226.672762] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11231.495896] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11242.898623] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11250.848519] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11259.649217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11260.188902] Lustre: lustre-OST0000: new disk, initializing [11260.192542] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11260.197709] Lustre: 318927:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11260.278753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11266.678703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11266.695823] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11266.901976] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11269.327725] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11281.628703] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11287.822407] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11288.121476] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11293.494809] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11293.728933] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11297.776290] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11297.786591] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11297.786845] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11297.792531] Lustre: Skipped 1 previous similar message [11297.816068] LustreError: Skipped 1 previous similar message [11301.944558] Lustre: server umount lustre-OST0000 complete [11308.002879] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11308.008032] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11308.011460] Lustre: Skipped 1 previous similar message [11308.016802] Lustre: Skipped 2 previous similar messages [11311.011537] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11311.032439] Lustre: Skipped 1 previous similar message [11312.549942] Lustre: server umount lustre-MDT0000 complete [11317.243825] LustreError: 317752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787624890 with bad export cookie 6917270748152351777 [11317.247521] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11317.258959] LustreError: 317752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11317.816746] Lustre: server umount lustre-MDT0001 complete [11322.022020] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11324.133174] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 22:28:14 (1787624894) [11331.399320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11331.785429] LustreError: 321240:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11331.808581] LustreError: 321240:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11331.860870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11336.525813] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11337.194606] LustreError: 321241:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11341.283696] LustreError: 321240:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11345.571419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11345.918019] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11350.694294] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11359.384420] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11366.087670] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11374.234524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11374.553396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11382.531457] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11393.561377] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11404.888991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11405.202975] Lustre: lustre-OST0001: new disk, initializing [11405.211139] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11405.231995] Lustre: 324077:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11405.432738] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11411.147460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11411.176405] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11411.287921] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11412.345895] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11412.386809] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11413.482548] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11413.498877] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11413.578931] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11415.522716] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11415.558568] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11419.685993] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11419.695823] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11425.869510] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11426.306521] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11428.017165] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11428.034615] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11434.815301] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ef8c82bd000.ost_server_uuid 50 [11436.640542] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ef8c82bd000.ost_server_uuid in FULL state after 0 sec [11443.678991] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ef8c82bd000.ost_server_uuid 50 [11444.583537] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11444.598677] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [11445.471707] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9ef8c82bd000.ost_server_uuid in IDLE state after 0 sec [11447.813673] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11462.631477] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11462.646470] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11462.652118] Lustre: Skipped 1 previous similar message [11467.689610] Lustre: server umount lustre-OST0001 complete [11467.763457] LustreError: 323062:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11467.778822] LustreError: 323062:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11472.869696] LustreError: 323063:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11472.894716] LustreError: 323063:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11477.991962] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11477.993518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11478.015772] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11478.027744] Lustre: Skipped 2 previous similar messages [11478.039047] Lustre: Skipped 2 previous similar messages [11480.857917] Lustre: server umount lustre-OST0000 complete [11488.227909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11488.242090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11488.249190] Lustre: Skipped 1 previous similar message [11490.955780] Lustre: server umount lustre-MDT0000 complete [11493.346500] LustreError: 323080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11493.372747] LustreError: 323080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11495.330464] LustreError: 321222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787625068 with bad export cookie 6917270748152352659 [11495.340296] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11495.360571] LustreError: 321222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11495.779885] Lustre: server umount lustre-MDT0001 complete [11508.969484] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [11512.011313] Key type lgssc unregistered [11512.383686] LNet: 326774:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11512.408799] LNetError: 326774:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11512.429166] LNet: Removed LNI 192.168.203.116@tcp [11513.392589] Key type .llcrypt unregistered [11513.395468] Key type ._llcrypt unregistered [11539.634470] Key type ._llcrypt registered [11539.642423] Key type .llcrypt registered [11539.763050] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11541.202858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11541.238122] alg: No test for adler32 (adler32-zlib) [11542.362318] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [11542.697298] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [11544.448324] Key type lgssc registered [11545.693719] Lustre: Echo OBD driver; http://www.lustre.org/ [11553.414910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11553.433905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11555.023752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11559.523527] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11564.524396] LustreError: 328196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11564.537975] LustreError: 328196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11568.176602] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11569.635428] LustreError: 328197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11574.754345] LustreError: 328196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11575.820217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11576.192484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11580.307080] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11588.531273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11588.773539] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11595.697292] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11600.068943] LustreError: 329875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11600.095889] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11606.506423] LustreError: 330359:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11606.518204] LustreError: 330359:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11607.480191] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11616.748051] LustreError: 329876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11616.765463] LustreError: 329876:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11617.412884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11617.725742] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11622.918666] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11625.492304] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11633.908569] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11642.344466] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11642.572845] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11761.123704] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11761.135667] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11761.139042] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11761.148170] Lustre: Skipped 1 previous similar message [11766.256200] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11766.263109] Lustre: Skipped 1 previous similar message [11771.366930] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11771.378882] Lustre: Skipped 1 previous similar message [11772.387356] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11772.709414] Lustre: server umount lustre-OST0001 complete [11776.496362] LustreError: 329875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11776.534538] LustreError: 329875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11784.674232] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11784.691472] LustreError: Skipped 1 previous similar message [11784.700410] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11784.710078] Lustre: Skipped 1 previous similar message [11784.713112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11785.702731] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11785.716746] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11787.954347] Lustre: server umount lustre-OST0000 complete [11791.842177] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11791.862250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11791.867345] Lustre: Skipped 3 previous similar messages [11796.454808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11796.467384] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11802.082270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11802.094957] Lustre: Skipped 3 previous similar messages [11806.176238] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11806.460033] Lustre: server umount lustre-MDT0000 complete [11810.381782] LustreError: 331884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787625383 with bad export cookie 2592847978951325991 [11810.392836] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11810.404048] LustreError: 331884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11810.823368] Lustre: server umount lustre-MDT0001 complete [11823.123272] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [11826.013387] Key type lgssc unregistered [11826.365879] LNet: 333152:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11826.389699] LNetError: 333152:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11826.418232] LNet: Removed LNI 192.168.203.116@tcp [11827.555616] Key type .llcrypt unregistered [11827.561708] Key type ._llcrypt unregistered [11875.709594] Key type ._llcrypt registered [11875.719663] Key type .llcrypt registered [11875.842930] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [11876.812801] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11876.870860] alg: No test for adler32 (adler32-zlib) [11877.921189] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [11878.114848] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [11879.832173] Key type lgssc registered [11881.122594] Lustre: Echo OBD driver; http://www.lustre.org/ [11890.305328] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11890.321690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11891.727839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11896.377219] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11901.925132] LustreError: 335444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11901.954279] LustreError: 335444:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11905.412488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11905.759558] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11910.139160] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11913.344194] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11920.438220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11920.885070] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11927.811590] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11931.120691] LustreError: 336932:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11936.191575] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11936.231046] LustreError: 336933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11936.257637] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11936.465743] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11941.873888] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11942.191507] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [11950.734735] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11956.966473] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11972.587243] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11972.591081] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11972.593330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11972.625442] Lustre: Skipped 1 previous similar message [11977.426892] Lustre: server umount lustre-MDT0000 complete [11977.703213] LustreError: 335439:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11977.727869] LustreError: 335439:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11981.278840] LustreError: 335425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787625554 with bad export cookie 13342710951513534852 [11981.288451] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11981.294913] LustreError: 335425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11981.543862] Lustre: server umount lustre-MDT0001 complete [11995.941644] Lustre: server umount lustre-OST0000 complete [11998.944477] Lustre: 334709:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787625555/real 1787625555] req@ffff8d484e75c700 x1874461143918976/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787625571 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11998.972327] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11998.986345] Lustre: Skipped 1 previous similar message [12000.287943] Lustre: server umount lustre-OST0001 complete [12018.701462] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12029.627733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12030.117968] LustreError: 340706:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12030.240436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12034.522919] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12040.674312] LustreError: 340707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12040.703364] LustreError: 340707:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12043.408543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12043.685482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12048.658986] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12052.317684] Lustre: 341846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12060.256063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12060.680876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12068.759692] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12068.853210] LustreError: 342199:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12076.039151] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12079.879177] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12085.775626] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12087.627372] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12096.892471] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12102.162219] Lustre: 343717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12110.315859] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12110.320242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12110.327771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12110.330827] Lustre: Skipped 3 previous similar messages [12111.357831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12113.470031] Lustre: server umount lustre-MDT0000 complete [12116.456318] LustreError: 340707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12116.489665] LustreError: 340707:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12117.619731] LustreError: 340686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787625690 with bad export cookie 13342710951513536553 [12117.627703] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12118.542163] Lustre: server umount lustre-MDT0001 complete [12134.827614] Lustre: server umount lustre-OST0000 complete [12137.955096] Lustre: 334707:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787625694/real 1787625694] req@ffff8d4752a1f100 x1874461143971968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787625710 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12138.002069] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12138.028976] Lustre: Skipped 3 previous similar messages [12139.089469] Lustre: server umount lustre-OST0001 complete [12144.164651] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12146.697910] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12148.573130] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:41:59 (1787625719) [12173.330782] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12181.811492] loop0: detected capacity change from 0 to 1099511627776 [12182.269848] SGI XFS with ACLs, security attributes, quota, no debug enabled [12182.281451] *** VALIDATE xfs *** [12182.301103] XFS (loop0): Mounting V5 Filesystem [12182.355740] XFS (loop0): Ending clean mount [12183.789434] loop1: detected capacity change from 0 to 18691697672192 [12280.923675] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [12290.696668] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12326.527347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12326.809628] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12326.834516] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12326.931625] Lustre: lustre-MDT0000: new disk, initializing [12327.008982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12327.014381] Lustre: Skipped 1 previous similar message [12327.029738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12330.999451] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12341.076666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12341.267968] Lustre: 348849:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12341.342074] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12341.349258] Lustre: Skipped 1 previous similar message [12341.436546] Lustre: lustre-MDT0001: new disk, initializing [12341.583282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12341.603758] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12346.248154] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12355.476193] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12360.962301] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12368.412640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12368.889620] Lustre: lustre-OST0000: new disk, initializing [12368.897379] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12368.902974] Lustre: 350818:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12370.432555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12370.448774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12370.631171] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12375.566562] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12385.750988] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12391.081369] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12391.314248] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12395.129315] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12395.255995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12401.123234] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12401.140915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12401.156437] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12401.165803] Lustre: Skipped 3 previous similar messages [12406.756900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12406.761125] Lustre: Skipped 2 previous similar messages [12411.361800] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12411.552409] Lustre: server umount lustre-OST0000 complete [12416.995092] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12417.011676] Lustre: Skipped 1 previous similar message [12417.038060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12430.304453] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12430.546091] Lustre: server umount lustre-MDT0000 complete [12431.361540] LustreError: 349728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12431.387563] LustreError: 349728:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12434.272147] LustreError: 348843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787626007 with bad export cookie 13342710951513538107 [12434.273850] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12434.280857] LustreError: 348843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12434.697936] Lustre: server umount lustre-MDT0001 complete [12447.490858] XFS (loop0): Unmounting Filesystem [12452.823841] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:47:03 (1787626023) [12459.353263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12459.828592] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12459.835271] Lustre: Skipped 2 previous similar messages [12464.032241] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12473.777603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12479.130365] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12488.307538] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12495.203444] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12502.375523] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12511.815519] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12518.823706] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12528.610869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12528.628122] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12528.633236] Lustre: Skipped 1 previous similar message [12528.638293] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12528.642796] LustreError: Skipped 1 previous similar message [12528.666895] Lustre: Skipped 5 previous similar messages [12532.601533] Lustre: server umount lustre-OST0000 complete [12543.395954] Lustre: server umount lustre-MDT0000 complete [12547.708192] LustreError: 353363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787626120 with bad export cookie 13342710951513538989 [12547.712734] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12547.720128] LustreError: 353363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12548.216813] Lustre: server umount lustre-MDT0001 complete [12559.228674] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [12562.229092] Key type lgssc unregistered [12562.587647] LNet: 357013:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12562.591490] LNetError: 357013:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12562.609396] LNet: Removed LNI 192.168.203.116@tcp [12563.400668] Key type .llcrypt unregistered [12563.408415] Key type ._llcrypt unregistered [12593.489407] Key type ._llcrypt registered [12593.493667] Key type .llcrypt registered [12593.574811] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [12594.374462] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12594.468763] alg: No test for adler32 (adler32-zlib) [12595.563136] Lustre: Lustre: Build Version: 2.17.57_80_g74d5152 [12595.817612] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [12597.497371] Key type lgssc registered [12598.696147] Lustre: Echo OBD driver; http://www.lustre.org/ [12608.294136] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12608.327978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12609.650980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12614.863600] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12619.235666] LustreError: 358582:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12619.276490] LustreError: 358582:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12624.353912] LustreError: 358581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12624.677748] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12625.052231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12629.638272] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12636.402508] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12642.057230] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12648.289792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12648.595549] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12654.764886] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12661.993957] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12670.437804] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12670.464271] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12670.471877] Lustre: Skipped 1 previous similar message [12674.928456] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.16@tcp (stopping) [12676.578350] Lustre: server umount lustre-MDT0001 complete [12680.028888] LustreError: 358575:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12685.142722] LustreError: 359279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12685.170257] LustreError: 359279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12687.877205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12688.024945] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12688.032523] Lustre: Found index 1 for lustre-MDT0001, updating log [12688.087546] Lustre: 358570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12688.406134] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12692.816515] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [12693.505782] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12698.097877] LustreError: 362181:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12698.116676] LustreError: 362181:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12698.134535] LustreError: 358574:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12698.206180] LustreError: 362184:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -17 [12698.216702] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12698.216702] [13000.688633] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13000.691127] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13000.708703] Lustre: Skipped 1 previous similar message [13000.710883] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13000.737482] Lustre: Skipped 3 previous similar messages [13004.843078] Lustre: server umount lustre-MDT0001 complete [13005.806741] LustreError: 358577:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13005.834184] LustreError: 358577:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13016.044038] LustreError: 358577:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13016.065511] LustreError: 358577:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13016.872438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13016.931799] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13016.935322] Lustre: Found index 1 for lustre-MDT0001, updating log [13016.968280] Lustre: 358570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13017.324054] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13020.885450] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13022.704923] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13025.272855] LustreError: 370717:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13025.277552] LustreError: 370717:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13025.293887] LustreError: 358574:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13025.299450] LustreError: 358574:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13025.419214] LustreError: 370720:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13025.424820] LustreError: 370720:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13025.429781] LustreError: 370720:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -17 [13025.438819] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.116@tcp [13025.438819] [13329.900964] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13329.904472] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13329.915182] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13329.942135] Lustre: Skipped 2 previous similar messages [13329.951736] Lustre: Skipped 1 previous similar message [13333.617817] Lustre: server umount lustre-MDT0001 complete [13335.015585] LustreError: 365671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13335.032042] LustreError: 365671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13343.850182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13343.965607] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13343.968714] Lustre: Found index 1 for lustre-MDT0001, updating log [13344.000578] Lustre: 358570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13344.309187] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13348.628764] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13349.366057] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13349.394740] LustreError: 379371:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13349.402478] LustreError: 379371:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13349.424587] LustreError: 358574:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13349.435860] LustreError: 358574:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13349.511600] LustreError: 379385:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13349.539074] LustreError: 379385:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.116@tcp: cfg command failed: rc = -114 [13349.548175] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13349.548175] [13671.905049] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13671.911354] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13671.915568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13671.925586] Lustre: Skipped 1 previous similar message [13671.953146] Lustre: Skipped 4 previous similar messages [13682.162584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13682.181132] Lustre: Skipped 5 previous similar messages [13686.240130] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13686.442446] Lustre: server umount lustre-MDT0000 complete [13687.278475] LustreError: 365671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13687.310921] LustreError: 365671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [13690.362113] LustreError: 358561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787627263 with bad export cookie 7557215629243211419 [13690.364347] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13690.373436] LustreError: 358561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13692.391516] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13692.406805] Lustre: Skipped 1 previous similar message [13697.080500] Lustre: server umount lustre-MDT0001 complete [13701.405543] Lustre: server umount lustre-OST0000 complete [13711.296886] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [13722.637911] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [13764.665840] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 23:08:54 (1787627334) [13773.516446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13773.851211] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13773.897764] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13774.018490] Lustre: lustre-MDT0000: new disk, initializing [13774.180552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13774.220644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13779.600816] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13790.604391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13790.761360] Lustre: 390123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13790.834560] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13790.840186] Lustre: Skipped 1 previous similar message [13790.911261] Lustre: lustre-MDT0001: new disk, initializing [13790.991088] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13791.036113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13791.042282] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13796.031696] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13806.510955] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13813.292591] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13820.792982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13821.115678] Lustre: lustre-OST0000: new disk, initializing [13821.120465] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13821.125953] Lustre: 392089:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13821.210410] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13827.746427] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [13830.199976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13830.209280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13830.329525] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13837.467346] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13845.988217] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13845.998553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13846.010755] Lustre: Skipped 1 previous similar message [13846.037521] Lustre: Skipped 4 previous similar messages [13847.434379] Lustre: server umount lustre-MDT0000 complete [13851.064646] LustreError: 390114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787627423 with bad export cookie 7557215629243213372 [13851.068938] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13851.072163] LustreError: 390114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13851.077793] LustreError: 390928:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13851.109242] LustreError: 390129:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13851.131020] LustreError: 390129:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13851.453379] Lustre: server umount lustre-MDT0001 complete [13855.674967] Lustre: server umount lustre-OST0000 complete [13887.784859] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [13899.436746] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [13931.928738] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13608 sec ========= 23:11:42 (1787627502) [13933.584334] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 23:11:44 (1787627504) === [13937.472919] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 23:11:48 (1787627508) === [13981.006522] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [13983.509343] Key type lgssc unregistered [13983.796217] LNet: 398456:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13983.804657] LNetError: 398456:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13983.845842] LNet: Removed LNI 192.168.203.116@tcp [13984.788478] Key type .llcrypt unregistered [13984.795015] Key type ._llcrypt unregistered