[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 506068982 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002473] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.005010] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.008024] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010188] LSM: Security Framework initializing [ 0.011048] Yama: becoming mindful. [ 0.012000] SELinux: Initializing. [ 0.012075] *** VALIDATE selinux *** [ 0.021504] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026170] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027201] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028108] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029129] *** VALIDATE tmpfs *** [ 0.031476] *** VALIDATE proc *** [ 0.033094] *** VALIDATE cgroup *** [ 0.034018] *** VALIDATE cgroup2 *** [ 0.036024] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037219] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039028] Spectre V2 : User space: Vulnerable [ 0.041004] Speculative Store Bypass: Vulnerable [ 0.043775] debug: unmapping init [mem 0xffffffff89459000-0xffffffff89460fff] [ 0.046164] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047747] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048030] ... version: 2 [ 0.049022] ... bit width: 48 [ 0.050018] ... generic registers: 4 [ 0.051027] ... value mask: 0000ffffffffffff [ 0.052023] ... max period: 00007fffffffffff [ 0.053022] ... fixed-purpose events: 3 [ 0.054020] ... event mask: 000000070000000f [ 0.056301] rcu: Hierarchical SRCU implementation. [ 0.058745] smp: Bringing up secondary CPUs ... [ 0.059726] x86: Booting SMP configuration: [ 0.060049] .... node #0, CPUs: #1 #2 #3 [ 0.067123] smp: Brought up 1 node, 4 CPUs [ 0.069018] smpboot: Max logical packages: 1 [ 0.070025] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.187617] node 0 deferred pages initialised in 115ms [ 0.192009] devtmpfs: initialized [ 0.193462] x86/mm: Memory block size: 128MB [ 0.197000] gcov: version magic: 0x41383552 [ 0.201338] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.205155] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.208445] pinctrl core: initialized pinctrl subsystem [ 0.210363] [ 0.211023] ************************************************************* [ 0.213018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.215017] ** ** [ 0.217018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.220140] ** ** [ 0.223020] ** This means that this kernel is built to expose internal ** [ 0.226018] ** IOMMU data structures, which may compromise security on ** [ 0.229017] ** your system. ** [ 0.232027] ** ** [ 0.235019] ** If you see this message and you are not debugging the ** [ 0.238015] ** kernel, report this immediately to your vendor! ** [ 0.241017] ** ** [ 0.244018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.247017] ************************************************************* [ 0.251106] NET: Registered protocol family 16 [ 0.255528] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.260580] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.264088] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.268017] cpuidle: using governor menu [ 0.269995] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.272753] PCI: Using configuration type 1 for base access [ 0.275159] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.284136] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.286048] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.290095] cryptd: max_cpu_qlen set to 1000 [ 0.295389] ACPI: Added _OSI(Module Device) [ 0.296017] ACPI: Added _OSI(Processor Device) [ 0.298018] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.300019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.304984] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.311263] ACPI: Interpreter enabled [ 0.313085] ACPI: PM: (supports S0 S3 S4 S5) [ 0.314014] ACPI: Using IOAPIC for interrupt routing [ 0.316236] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.320435] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.329885] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.332058] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.334019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.338088] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.344630] acpiphp: Slot [2] registered [ 0.346202] acpiphp: Slot [5] registered [ 0.347134] acpiphp: Slot [6] registered [ 0.349128] acpiphp: Slot [7] registered [ 0.350108] acpiphp: Slot [8] registered [ 0.352100] acpiphp: Slot [9] registered [ 0.353087] acpiphp: Slot [10] registered [ 0.354097] acpiphp: Slot [3] registered [ 0.356097] acpiphp: Slot [4] registered [ 0.357150] acpiphp: Slot [11] registered [ 0.359127] acpiphp: Slot [12] registered [ 0.360088] acpiphp: Slot [13] registered [ 0.362174] acpiphp: Slot [14] registered [ 0.363176] acpiphp: Slot [15] registered [ 0.365103] acpiphp: Slot [16] registered [ 0.367100] acpiphp: Slot [17] registered [ 0.368163] acpiphp: Slot [18] registered [ 0.370103] acpiphp: Slot [19] registered [ 0.371113] acpiphp: Slot [20] registered [ 0.373205] acpiphp: Slot [21] registered [ 0.375108] acpiphp: Slot [22] registered [ 0.377173] acpiphp: Slot [23] registered [ 0.378102] acpiphp: Slot [24] registered [ 0.380173] acpiphp: Slot [25] registered [ 0.382099] acpiphp: Slot [26] registered [ 0.383229] acpiphp: Slot [27] registered [ 0.385115] acpiphp: Slot [28] registered [ 0.387154] acpiphp: Slot [29] registered [ 0.389107] acpiphp: Slot [30] registered [ 0.391109] acpiphp: Slot [31] registered [ 0.392067] PCI host bridge to bus 0000:00 [ 0.394021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.396022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.398026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.401026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.404030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.408027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.410193] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.413706] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.418627] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.428016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.433583] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.436017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.439019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.442022] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.445703] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.448836] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.451191] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.454746] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.460015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.472017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.476014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.517114] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.534021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.548022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.577016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.593107] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.608016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.627017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.660017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.674616] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.687024] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.695019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.719020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.734046] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.748017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.756015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.781019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.798413] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.806018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.818019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.848017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.870393] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.879016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.889017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.912018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.924517] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.927361] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.930461] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.932308] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.934218] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.938039] iommu: Default domain type: Passthrough [ 0.940368] SCSI subsystem initialized [ 0.941151] ACPI: bus type USB registered [ 0.943091] usbcore: registered new interface driver usbfs [ 0.945091] usbcore: registered new interface driver hub [ 0.946108] usbcore: registered new device driver usb [ 0.948175] pps_core: LinuxPPS API ver. 1 registered [ 0.950011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.953061] PTP clock support registered [ 0.955417] EDAC MC: Ver: 3.0.0 [ 0.957035] PCI: Using ACPI for IRQ routing [ 0.959890] NetLabel: Initializing [ 0.961029] NetLabel: domain hash size = 128 [ 0.963011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.964118] NetLabel: unlabeled traffic allowed by default [ 0.967491] vgaarb: loaded [ 0.969445] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.972028] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.982803] clocksource: Switched to clocksource kvm-clock [ 1.101111] VFS: Disk quotas dquot_6.6.0 [ 1.102516] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.104881] *** VALIDATE ramfs *** [ 1.106079] *** VALIDATE hugetlbfs *** [ 1.107364] pnp: PnP ACPI init [ 1.109934] pnp: PnP ACPI: found 6 devices [ 1.139267] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.142510] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.144456] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.146782] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.149344] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.152302] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.155719] NET: Registered protocol family 2 [ 1.158613] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.163883] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.167807] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.173128] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.176419] TCP: Hash tables configured (established 65536 bind 65536) [ 1.178819] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.181542] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.183781] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.186418] NET: Registered protocol family 1 [ 1.189489] RPC: Registered named UNIX socket transport module. [ 1.191719] RPC: Registered udp transport module. [ 1.193172] RPC: Registered tcp transport module. [ 1.195014] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.196921] NET: Registered protocol family 44 [ 1.198167] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.199923] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.201632] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.203609] PCI: CLS 0 bytes, default 64 [ 1.205601] Unpacking initramfs... [ 2.740177] debug: unmapping init [mem 0xffff9f31bcc54000-0xffff9f31bffbffff] [ 2.742584] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.744489] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.746478] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.316420] Initialise system trusted keyrings [ 3.318255] Key type blacklist registered [ 3.320963] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.328255] zbud: loaded [ 3.332159] *** VALIDATE nfs *** [ 3.333556] *** VALIDATE nfs4 *** [ 3.337647] pstore: using deflate compression [ 3.343418] Platform Keyring initialized [ 3.448754] NET: Registered protocol family 38 [ 3.451263] Key type asymmetric registered [ 3.452980] Asymmetric key parser 'x509' registered [ 3.455847] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.461506] io scheduler mq-deadline registered [ 3.463245] io scheduler kyber registered [ 3.465064] io scheduler bfq registered [ 3.467090] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.471710] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.474857] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.477935] ACPI: Power Button [PWRF] [ 3.483559] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.491076] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.508554] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.521787] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.551291] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.581928] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.613255] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.618115] Non-volatile memory driver v1.3 [ 3.619776] Linux agpgart interface v0.103 [ 3.662541] virtio_blk virtio1: [vda] 150408 512-byte logical blocks (77.0 MB/73.4 MiB) [ 3.665611] vda: detected capacity change from 0 to 77008896 [ 3.682594] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.685288] vdb: detected capacity change from 0 to 1073741824 [ 3.700557] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.702965] vdc: detected capacity change from 0 to 2621440000 [ 3.718774] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.721361] vdd: detected capacity change from 0 to 2621440000 [ 3.737493] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.740808] vde: detected capacity change from 0 to 4294967296 [ 3.756090] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.759563] vdf: detected capacity change from 0 to 4294967296 [ 3.765857] libphy: Fixed MDIO Bus: probed [ 3.771171] usbcore: registered new interface driver usbserial_generic [ 3.773652] usbserial: USB Serial support registered for generic [ 3.776377] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.780282] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.782087] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.784830] mousedev: PS/2 mouse device common for all mice [ 3.787748] rtc_cmos 00:05: RTC can wake from S4 [ 3.791849] rtc_cmos 00:05: registered as rtc0 [ 3.793948] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.798412] intel_pstate: CPU model not supported [ 3.801772] hid: raw HID events driver (C) Jiri Kosina [ 3.802572] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.804258] usbcore: registered new interface driver usbhid [ 3.812022] usbhid: USB HID core driver [ 3.813408] drop_monitor: Initializing network drop monitor service [ 3.815461] Initializing XFRM netlink socket [ 3.818193] NET: Registered protocol family 10 [ 3.821112] Segment Routing with IPv6 [ 3.822525] NET: Registered protocol family 17 [ 3.827536] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.835478] mpls_gso: MPLS GSO support [ 3.850453] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.863883] RAS: Correctable Errors collector initialized. [ 3.867600] AVX version of gcm_enc/dec engaged. [ 3.869951] AES CTR mode by8 optimization enabled [ 4.008438] sched_clock: Marking stable (4008406041, 0)->(4922258892, -913852851) [ 4.017960] registered taskstats version 1 [ 4.025911] Loading compiled-in X.509 certificates [ 4.033138] zswap: loaded using pool lzo/zbud [ 4.112431] Key type big_key registered [ 4.158861] Key type encrypted registered [ 4.161778] ima: No TPM chip found, activating TPM-bypass! [ 4.165479] ima: Allocated hash algorithm: sha1 [ 4.169873] ima: No architecture policies found [ 4.176930] evm: Initialising EVM extended attributes: [ 4.187190] evm: security.selinux [ 4.191260] evm: security.ima [ 4.192250] evm: security.capability [ 4.194720] evm: HMAC attrs: 0x1 [ 4.207836] rtc_cmos 00:05: setting system clock to 2026-09-22 17:01:12 UTC (1790096472) [ 4.228876] debug: unmapping init [mem 0xffffffff8a403000-0xffffffff8a5fffff] [ 4.252329] debug: unmapping init [mem 0xffffffff89182000-0xffffffff89458fff] [ 4.265651] Write protecting the kernel read-only data: 28672k [ 4.275559] debug: unmapping init [mem 0xffffffff87803000-0xffffffff879fffff] [ 4.282255] debug: unmapping init [mem 0xffffffff88114000-0xffffffff881fffff] [ 4.358390] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.379759] systemd[1]: Detected virtualization kvm. [ 4.384017] systemd[1]: Detected architecture x86-64. [ 4.391269] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.476774] systemd[1]: No hostname configured. [ 4.481274] systemd[1]: Set hostname to . [ 4.486526] random: systemd: uninitialized urandom read (16 bytes read) [ 4.507393] systemd[1]: Initializing machine ID from random generator. [ 4.900462] random: systemd: uninitialized urandom read (16 bytes read) [ 4.905916] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.922413] random: systemd: uninitialized urandom read (16 bytes read) [ 4.927550] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.942933] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.260550] device-mapper: uevent: version 1.0.3 [ 7.265918] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization.[ 9.629373] random: fast init done [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 9.957155] virtio_net virtio0 ens2: renamed from eth0 [ 10.197892] scsi host0: ata_piix [ 10.248023] scsi host1: ata_piix [ 10.249734] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.261111] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.713869] random: crng init done [ 15.715958] random: 7 urandom warning(s) missed due to ratelimiting [ 16.434245] dracut-initqueue[584]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 18.947637] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 22.050366] printk: systemd: 26 output lines suppressed due to ratelimiting [ 22.942263] SELinux: Disabled at runtime. [ 23.136509] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 23.174349] systemd[1]: Detected virtualization kvm. [ 23.181196] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 24.905846] systemd[1]: initrd-switch-root.service: Succeeded. [ 24.914261] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 24.934749] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 24.954038] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 24.965192] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 24.985449] systemd[1]: Starting Journal Service... Starting Journal Service... [ 25.006207] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Remount Root and Kernel File Systems... Starting udev Coldplug all Devices... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Reached target RPC Port Mapper. [ OK [ 25.476646] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 26.667595] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.850784] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.066789] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 28.456792] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 28.619495] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit)[ 33.037785] Key type dns_resolver registered [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 33.819044] NFS: Registering the id_resolver key type [ 33.829581] Key type id_resolver registered [ 33.838354] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg322-server login: [ 58.011205] hrtimer: interrupt took 5187631 ns [ 93.927583] libcfs: loading out-of-tree module taints kernel. [ 94.019802] Key type ._llcrypt registered [ 94.025282] Key type .llcrypt registered [ 94.168940] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 113.652904] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 115.260739] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 115.280620] alg: No test for adler32 (adler32-zlib) [ 116.838722] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 117.733662] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 119.527204] Key type lgssc registered [ 121.188342] Lustre: Echo OBD driver; http://www.lustre.org/ [ 138.975290] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 181.144615] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 194.231517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 194.257368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 195.533865] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 195.564612] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 195.673533] Lustre: lustre-MDT0000: new disk, initializing [ 195.758059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.769612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 200.564925] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 214.576337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.656569] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 214.689384] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 214.693603] Lustre: Skipped 1 previous similar message [ 214.821213] Lustre: lustre-MDT0001: new disk, initializing [ 214.919075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 214.969302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 214.975350] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 219.442994] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.696166] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 234.384979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.622955] Lustre: lustre-OST0000: new disk, initializing [ 234.633888] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 234.638880] Lustre: 8447:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.713874] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 241.162811] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 241.700063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 241.713995] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 241.783645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 256.608324] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 256.747911] Lustre: lustre-OST0001: new disk, initializing [ 256.757581] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 256.766384] Lustre: 9520:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 256.833450] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 262.940915] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 264.248490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 264.259061] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 264.368768] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 276.080844] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 284.305471] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 290.628951] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing check_logdir /tmp/testlogs/ [ 295.064366] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing yml_node [ 300.435365] Lustre: DEBUG MARKER: Client: 2.17.58.63 [ 303.200871] Lustre: DEBUG MARKER: MDS: 2.17.58.63 [ 305.539816] Lustre: DEBUG MARKER: OSS: 2.17.58.63 [ 307.155438] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 22 13:06:13 EDT 2026 [ 324.896610] Lustre: DEBUG MARKER: only running tests: 76-120 [ 327.010349] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 328.976248] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 346.594179] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 346.599333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 346.613079] Lustre: Skipped 1 previous similar message [ 351.753193] Lustre: server umount lustre-MDT0000 complete [ 355.527455] LustreError: 6502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790096823 with bad export cookie 9675809062888879559 [ 355.528813] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 355.533961] LustreError: 6502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 356.834502] LustreError: 6515:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 356.835395] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 356.842086] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 356.842094] Lustre: Skipped 3 previous similar messages [ 356.850353] LustreError: 6515:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 356.866745] Lustre: Skipped 3 previous similar messages [ 360.867494] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 360.867851] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 360.885279] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 360.895562] Lustre: Skipped 2 previous similar messages [ 361.985461] Lustre: server umount lustre-MDT0001 complete [ 365.912115] Lustre: server umount lustre-OST0000 complete [ 369.651891] Lustre: server umount lustre-OST0001 complete [ 376.435841] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 387.255664] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 425.535415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 425.734614] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 425.784369] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 425.928858] Lustre: lustre-MDT0000: new disk, initializing [ 426.085057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 426.123678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 431.643312] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 443.099915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 443.229655] Lustre: 15120:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 443.311964] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 443.313747] Lustre: Skipped 1 previous similar message [ 443.355359] Lustre: lustre-MDT0001: new disk, initializing [ 443.422189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 443.504709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 443.531343] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 448.330561] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 460.044200] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 467.909213] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 474.684644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 474.972339] Lustre: lustre-OST0000: new disk, initializing [ 474.976750] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 474.988811] Lustre: 17089:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 475.142641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 476.958275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 476.969069] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 477.125285] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 480.630903] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 490.147657] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 495.376341] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 495.593485] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 500.721209] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 501.035516] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 507.873361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 507.876721] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 507.912069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 509.673752] Lustre: server umount lustre-OST0000 complete [ 515.047522] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 515.062773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 515.078617] Lustre: Skipped 1 previous similar message [ 515.083238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 515.091081] Lustre: Skipped 1 previous similar message [ 523.234605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 523.245654] Lustre: Skipped 3 previous similar messages [ 528.354338] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 528.762694] Lustre: server umount lustre-MDT0000 complete [ 533.207857] LustreError: 15114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097001 with bad export cookie 9675809062888881092 [ 533.220915] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 533.223735] LustreError: 15114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 533.785477] Lustre: server umount lustre-MDT0001 complete [ 540.309688] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 13:10:06 (1790097006) [ 548.160735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 548.735819] LustreError: 19357:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 548.870152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 553.357799] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 553.955711] LustreError: 19358:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.075648] LustreError: 19357:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 563.048982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 568.466027] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 575.933662] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 583.645571] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 592.577101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 600.096755] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 607.305225] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 614.484563] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 624.097478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 624.106687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 624.121896] Lustre: Skipped 2 previous similar messages [ 624.137163] Lustre: Skipped 4 previous similar messages [ 628.129906] Lustre: server umount lustre-MDT0000 complete [ 629.229260] LustreError: 20085:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 629.269297] LustreError: 20085:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 632.188856] LustreError: 19337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097100 with bad export cookie 9675809062888881974 [ 632.193539] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 632.201483] LustreError: 19337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 632.551264] Lustre: server umount lustre-MDT0001 complete [ 647.408511] Lustre: server umount lustre-OST0000 complete [ 673.334486] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 684.269146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 684.586104] LustreError: 23764:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 684.650891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 684.660682] Lustre: Skipped 2 previous similar messages [ 689.614411] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 697.711868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 702.188200] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 705.675655] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 713.971123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 721.578406] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 729.839356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 730.025460] Lustre: lustre-OST0001: new disk, initializing [ 730.035139] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 730.048600] Lustre: 26080:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 731.214654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 731.230809] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 731.411082] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 737.477218] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 747.208110] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 750.599250] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 764.896708] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 764.900965] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 764.914904] Lustre: Skipped 1 previous similar message [ 764.924625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 769.744922] Lustre: server umount lustre-MDT0000 complete [ 772.588617] LustreError: 23764:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 772.610652] LustreError: 23764:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 773.856421] LustreError: 25282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097242 with bad export cookie 9675809062888882821 [ 773.863030] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 773.879750] LustreError: 25282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 774.351675] Lustre: server umount lustre-MDT0001 complete [ 787.980355] Lustre: server umount lustre-OST0000 complete [ 802.228859] Lustre: server umount lustre-OST0001 complete [ 820.857196] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 830.523409] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 831.046945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 831.054920] Lustre: Skipped 3 previous similar messages [ 835.501605] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 841.192588] LustreError: 29101:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 841.215854] LustreError: 29101:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 844.549718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 849.847827] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 853.133796] Lustre: 30247:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 859.311642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 866.136796] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 875.805343] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 882.970977] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 891.782064] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 896.150443] Lustre: 32124:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 906.210119] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 906.233393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 906.268222] Lustre: Skipped 3 previous similar messages [ 906.276730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 906.279732] Lustre: Skipped 4 previous similar messages [ 909.025845] Lustre: server umount lustre-MDT0000 complete [ 913.005538] LustreError: 29080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097381 with bad export cookie 9675809062888884116 [ 913.009601] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 913.019535] LustreError: 29080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 913.386359] Lustre: server umount lustre-MDT0001 complete [ 927.789994] Lustre: server umount lustre-OST0000 complete [ 942.151634] Lustre: server umount lustre-OST0001 complete [ 952.109602] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 13:16:58 (1790097418) [ 984.636434] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 994.374309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 994.843057] LustreError: 34932:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 994.854502] LustreError: 34932:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 999.405952] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1008.106418] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1012.889443] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1015.609654] Lustre: 36084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1022.384678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1029.485410] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1038.331355] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1044.931268] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1053.169360] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1057.008537] Lustre: 37963:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1064.929648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1064.944229] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1064.969615] Lustre: Skipped 3 previous similar messages [ 1064.980206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1064.984852] Lustre: Skipped 4 previous similar messages [ 1068.554265] Lustre: server umount lustre-MDT0000 complete [ 1071.864667] LustreError: 34914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097540 with bad export cookie 9675809062888885222 [ 1071.868161] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1071.876824] LustreError: 34914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1072.206967] Lustre: server umount lustre-MDT0001 complete [ 1086.131571] Lustre: server umount lustre-OST0000 complete [ 1100.667563] Lustre: server umount lustre-OST0001 complete [ 1109.697825] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 13:19:36 (1790097576) [ 1126.646378] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1138.313270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1138.938694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1138.947830] Lustre: Skipped 7 previous similar messages [ 1142.998753] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1153.101691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1158.207705] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1161.525291] Lustre: 41252:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1168.034297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1175.205398] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1185.750311] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1193.484376] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1201.964425] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1205.618624] Lustre: 43130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1206.607959] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1206.617270] Lustre: Skipped 1 previous similar message [ 1209.824567] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1209.838273] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1209.858209] Lustre: Skipped 3 previous similar messages [ 1216.055495] Lustre: server umount lustre-MDT0000 complete [ 1220.074113] LustreError: 40086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097688 with bad export cookie 9675809062888886328 [ 1220.080462] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1220.088570] LustreError: 40086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1234.399150] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1234.641390] Lustre: server umount lustre-MDT0001 complete [ 1243.629816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1244.831375] LustreError: 43993:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 1244.838286] LustreError: 43993:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff9f322c1ff480 x1877052318161792/t0(0) o250->MGC192.168.203.122@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1790097713 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1244.859874] LustreError: 43993:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 1245.664372] LustreError: 3642:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9f3205ba2300 x1877052318162944/t0(0) o250->MGC192.168.203.122@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1250.700104] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1251.303077] LustreError: 41616:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1251.328504] LustreError: 41616:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 31 previous similar messages [ 1259.898070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1261.358964] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1265.466634] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1272.804698] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1274.556894] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1280.818927] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1282.683150] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1291.238727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1294.526334] Lustre: server umount lustre-MDT0000 complete [ 1299.941996] LustreError: 40086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097768 with bad export cookie 9675809062888887420 [ 1299.959696] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1299.965353] LustreError: 40086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1301.184396] Lustre: server umount lustre-MDT0001 complete [ 1317.855161] Lustre: 3646:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790097770/real 1790097770] req@ffff9f322fe47100 x1877052318199296/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790097786 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1319.144702] Lustre: server umount lustre-OST0000 complete [ 1319.904552] Lustre: 3644:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790097772/real 1790097772] req@ffff9f3237e92a00 x1877052318199552/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790097788 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1323.012888] Lustre: server umount lustre-OST0001 complete [ 1332.592557] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 13:23:19 (1790097799) [ 1349.545989] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1360.619441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1365.234502] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1373.219604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1378.927894] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1381.818674] Lustre: 48897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1388.094154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1394.645364] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1402.331235] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1408.965280] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1416.250525] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1420.233892] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1432.077623] Lustre: server umount lustre-MDT0000 complete [ 1435.116950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1435.127488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1435.143066] Lustre: Skipped 10 previous similar messages [ 1435.457449] LustreError: 48499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790097903 with bad export cookie 9675809062888888246 [ 1435.465577] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1435.470920] LustreError: 48499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1435.782387] Lustre: server umount lustre-MDT0001 complete [ 1451.809049] Lustre: 3644:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790097904/real 1790097904] req@ffff9f3103a45c00 x1877052318251520/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790097920 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1456.095160] Lustre: 3646:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790097908/real 1790097908] req@ffff9f3102735880 x1877052318251904/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790097924 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1456.137585] Lustre: 3646:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1459.356214] Lustre: server umount lustre-OST0000 complete [ 1461.023121] Lustre: 3643:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790097913/real 1790097913] req@ffff9f3103a45880 x1877052318252416/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790097929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1461.049273] Lustre: 3643:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1463.064518] Lustre: server umount lustre-OST0001 complete [ 1471.815977] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 13:25:38 (1790097938) [ 1474.394418] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1476.360091] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 13:25:43 (1790097943) [ 1486.774262] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 1495.611933] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1517.452577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1517.732803] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1517.824854] Lustre: lustre-MDT0000: new disk, initializing [ 1517.936964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1522.532492] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1531.687212] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1537.675556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1537.935524] Lustre: lustre-OST0000: new disk, initializing [ 1537.939467] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1537.943549] Lustre: Skipped 1 previous similar message [ 1537.949519] Lustre: 54975:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1539.861477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1539.870561] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1539.908926] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1543.714399] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1551.585555] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1553.200890] Lustre: DEBUG MARKER: create test files [ 1581.038991] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1581.041104] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1581.046960] Lustre: Skipped 25 previous similar messages [ 1584.540763] Lustre: server umount lustre-OST0000 complete [ 1588.006771] Lustre: server umount lustre-MDT0000 complete [ 1598.703250] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1601.290571] Key type lgssc unregistered [ 1601.553576] LNet: 56603:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1601.569143] LNetError: 56603:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1601.585884] LNet: Removed LNI 192.168.203.122@tcp [ 1602.482779] Key type .llcrypt unregistered [ 1602.486938] Key type ._llcrypt unregistered [ 1626.608488] Key type ._llcrypt registered [ 1626.611529] Key type .llcrypt registered [ 1626.693316] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1627.514059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1627.557080] alg: No test for adler32 (adler32-zlib) [ 1628.565977] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 1628.775959] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1630.463244] Key type lgssc registered [ 1631.410697] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1638.079766] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1638.103660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1639.512074] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1642.657919] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1648.871920] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1654.532947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1654.842249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1659.845908] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1664.046876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1664.143071] Lustre: 59332:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1664.152506] Lustre: 59332:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1664.156542] Lustre: 59332:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1664.161374] Lustre: 59332:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1664.168608] Lustre: 59332:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1664.179967] Lustre: 59332:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1666.252833] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1670.622367] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1675.049588] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1684.453939] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1684.459456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1684.475401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1685.928123] Lustre: server umount lustre-OST0000 complete [ 1689.184533] Lustre: server umount lustre-MDT0000 complete [ 1697.977922] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1700.188877] Key type lgssc unregistered [ 1700.422412] LNet: 60940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1700.430936] LNetError: 60940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1700.443707] LNet: Removed LNI 192.168.203.122@tcp [ 1701.224158] Key type .llcrypt unregistered [ 1701.227056] Key type ._llcrypt unregistered [ 1725.195870] Key type ._llcrypt registered [ 1725.197788] Key type .llcrypt registered [ 1725.276385] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1725.943768] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1725.956398] alg: No test for adler32 (adler32-zlib) [ 1726.956711] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 1727.125313] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1728.767280] Key type lgssc registered [ 1729.634714] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1734.819437] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1734.834381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1736.093491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1738.899139] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1743.991927] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1749.444602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1749.687238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1753.903268] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1758.725085] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1760.934955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1760.975638] Lustre: 63765:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1760.984047] Lustre: 63765:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1760.990803] Lustre: 63765:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1760.998758] Lustre: 63765:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1761.006412] Lustre: 63765:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1761.009787] Lustre: 63765:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1767.975712] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1776.609270] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1776.610480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1776.622764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1779.726971] Lustre: server umount lustre-OST0000 complete [ 1782.800953] Lustre: server umount lustre-MDT0000 complete [ 1790.709672] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1792.672643] Key type lgssc unregistered [ 1792.892392] LNet: 65320:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1792.900403] LNetError: 65320:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1792.915564] LNet: Removed LNI 192.168.203.122@tcp [ 1793.456344] Key type .llcrypt unregistered [ 1793.458532] Key type ._llcrypt unregistered [ 1805.809240] Key type ._llcrypt registered [ 1805.811559] Key type .llcrypt registered [ 1805.898390] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 1816.952399] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1817.520151] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1817.602173] alg: No test for adler32 (adler32-zlib) [ 1818.569279] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 1818.718391] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1820.343361] Key type lgssc registered [ 1821.122875] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1846.336051] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1846.389065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1847.571887] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1847.612717] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1847.661666] Lustre: lustre-MDT0000: new disk, initializing [ 1847.703403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.723214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1850.422683] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1857.022024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1857.086839] Lustre: 68464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1857.130956] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1857.133946] Lustre: Skipped 1 previous similar message [ 1857.176420] Lustre: lustre-MDT0001: new disk, initializing [ 1857.238198] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1857.260252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1857.275100] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1859.837796] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1865.882738] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1869.431833] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1873.652906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1873.827149] Lustre: lustre-OST0000: new disk, initializing [ 1873.829889] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1873.833971] Lustre: 70433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1873.889594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1877.127283] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1879.580340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1879.588778] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1879.678788] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1882.782199] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1885.377984] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1885.502787] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1887.575477] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1887.675347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1889.760391] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1889.765267] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1889.774130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1890.272640] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1890.278252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1890.285456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1890.295959] Lustre: Skipped 1 previous similar message [ 1895.395035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1895.399817] Lustre: Skipped 1 previous similar message [ 1900.515022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1900.520997] Lustre: Skipped 1 previous similar message [ 1903.071214] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1903.136936] Lustre: server umount lustre-OST0000 complete [ 1905.634890] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1905.636656] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1905.642180] Lustre: Skipped 1 previous similar message [ 1911.176909] Lustre: server umount lustre-MDT0000 complete [ 1913.364189] LustreError: 68456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790098381 with bad export cookie 1231575946549441078 [ 1913.365462] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1913.375524] LustreError: 68456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1913.567903] Lustre: server umount lustre-MDT0001 complete [ 1918.405145] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 13:33:05 (1790098385) [ 1924.858212] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1947.944606] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 1954.481619] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1978.532351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1978.753216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1978.778434] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1978.830283] Lustre: lustre-MDT0000: new disk, initializing [ 1978.877190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1978.892619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1982.013936] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1989.043751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1989.127239] Lustre: 76444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1989.182332] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1989.185945] Lustre: Skipped 1 previous similar message [ 1989.243902] Lustre: lustre-MDT0001: new disk, initializing [ 1989.312498] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1989.345444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1989.356394] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1992.352898] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1998.942274] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2003.482772] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2008.795388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2008.990533] Lustre: lustre-OST0000: new disk, initializing [ 2008.996931] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2009.001337] Lustre: 78410:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2009.062739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2013.737385] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2015.254924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2015.263199] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2015.360982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2020.613988] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2023.395422] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2023.520933] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2026.358094] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2026.514472] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2030.563056] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2030.564771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2030.576741] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2030.582747] Lustre: Skipped 1 previous similar message [ 2030.587085] Lustre: Skipped 3 previous similar messages [ 2042.335114] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2042.449225] Lustre: server umount lustre-OST0000 complete [ 2045.920841] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2045.921418] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2051.040319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2051.044368] Lustre: Skipped 8 previous similar messages [ 2059.231166] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2059.339533] Lustre: server umount lustre-MDT0000 complete [ 2061.655963] LustreError: 76436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790098529 with bad export cookie 1231575946549441960 [ 2061.658934] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2061.670187] LustreError: 76436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2061.843511] Lustre: server umount lustre-MDT0001 complete [ 2067.564937] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 13:35:34 (1790098534) [ 2072.146807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2072.470337] LustreError: 80823:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2072.486431] LustreError: 80823:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2072.534015] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2075.436859] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2077.666052] LustreError: 80824:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2081.764092] LustreError: 80823:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2081.851643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2085.787327] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2091.618259] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2095.494519] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2100.402722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2104.722508] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2109.997649] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2112.451734] Lustre: 83491:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 5d82e4f8-8ddd-4cac-b793-bbaedce84841 at adminstrative request [ 2116.964128] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2116.976605] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2127.331122] Lustre: MGS: Client 5d82e4f8-8ddd-4cac-b793-bbaedce84841 (at 0@lo) reconnecting [ 2127.340547] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x11176fcc57feb91a to 0x11176fcc57febbba [ 2127.352695] Lustre: MGC192.168.203.122@tcp: Connection restored to 0@lo (at 0@lo) [ 2147.692404] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2147.810931] Lustre: lustre-OST0001: new disk, initializing [ 2147.817109] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2147.821334] Lustre: 83889:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2147.867487] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2147.872312] Lustre: Skipped 2 previous similar messages [ 2151.769107] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2157.616925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2157.629691] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2157.689739] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2158.655804] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2162.668477] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2162.672549] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2162.678814] Lustre: Skipped 2 previous similar messages [ 2162.679244] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2162.682314] Lustre: Skipped 3 previous similar messages [ 2166.760421] Lustre: server umount lustre-OST0001 complete [ 2167.785568] LustreError: 84321:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2167.810263] LustreError: 84321:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2172.904635] LustreError: 82642:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2172.925153] LustreError: 82642:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2175.744076] Lustre: server umount lustre-OST0000 complete [ 2179.553447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2179.563135] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2179.575128] Lustre: Skipped 2 previous similar messages [ 2184.512924] Lustre: server umount lustre-MDT0000 complete [ 2187.260257] LustreError: 80803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790098655 with bad export cookie 1231575946549443514 [ 2187.264854] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2187.273657] LustreError: 80803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2187.514399] Lustre: server umount lustre-MDT0001 complete [ 2193.054102] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 13:37:40 (1790098660) [ 2194.013392] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2195.111191] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 13:37:42 (1790098662) [ 2196.212394] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2197.560766] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 13:37:44 (1790098664) [ 2198.519720] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2199.709367] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 13:37:46 (1790098666) [ 2200.713598] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2201.851531] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 13:37:49 (1790098669) [ 2202.925425] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2206.529365] loop: module loaded [ 2206.544395] loop0: detected capacity change from 0 to 4294967296 [ 2206.571942] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2207.586029] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2209.680375] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2212.126251] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2229.176717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2229.210680] LustreError: 88007:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2229.215104] LustreError: 88007:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2229.218767] LustreError: 88007:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2229.222902] LustreError: 88007:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2240.999462] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 2247.717636] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2269.917267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2270.096359] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2270.113162] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2270.165413] Lustre: lustre-MDT0000: new disk, initializing [ 2270.224666] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2270.241687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2272.978728] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2279.796076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2279.860945] Lustre: 90869:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2280.016750] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2282.562744] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2288.976424] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2292.861533] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2297.495884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2297.668281] Lustre: 92836:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2302.142404] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2305.088733] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2305.197564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2309.207483] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2312.461334] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2312.612788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2315.806584] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2315.947728] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2320.871968] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2320.878747] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2320.882782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2320.890479] Lustre: Skipped 2 previous similar messages [ 2320.902311] Lustre: Skipped 6 previous similar messages [ 2323.371068] Lustre: server umount lustre-OST0000 complete [ 2340.319122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2340.464517] Lustre: server umount lustre-MDT0000 complete [ 2341.346286] LustreError: 90878:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.360766] LustreError: 90878:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2342.912452] LustreError: 90863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790098811 with bad export cookie 1231575946549444130 [ 2342.916262] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.917948] LustreError: 90863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2343.161508] Lustre: server umount lustre-MDT0001 complete [ 2349.238926] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 13:40:16 (1790098816) [ 2354.126531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2357.935696] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2359.776902] LustreError: 95250:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2359.800950] LustreError: 95250:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2364.391576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2368.295425] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2374.005203] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2377.911510] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2382.725627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2387.606185] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2392.621168] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2396.096411] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2400.446996] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 2403.752989] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2403.905573] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2408.111611] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2408.239209] Lustre: lustre-OST0001: new disk, initializing [ 2408.241654] Lustre: Skipped 2 previous similar messages [ 2408.244560] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2408.247550] Lustre: Skipped 3 previous similar messages [ 2408.251599] Lustre: 98643:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2408.326161] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2408.336438] Lustre: Skipped 5 previous similar messages [ 2412.450978] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2417.712866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2417.724446] Lustre: Skipped 2 previous similar messages [ 2417.726796] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2417.784670] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2418.874941] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2421.805685] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2421.940216] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2424.739685] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2424.900721] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2440.002431] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2460.421225] Lustre: Failing over lustre-MDT0000 [ 2460.748629] Lustre: server umount lustre-MDT0000 complete [ 2462.181783] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2462.191067] LustreError: Skipped 2 previous similar messages [ 2462.197262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2462.208482] Lustre: Skipped 2 previous similar messages [ 2462.211861] LustreError: 95250:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2462.227367] LustreError: 95250:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2479.871977] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2479.874250] LDISKFS-fs (dm-0): recovery complete [ 2479.880354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2479.991481] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2480.199065] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2482.863695] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2483.126774] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2483.127206] Lustre: 95245:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2485.230069] Lustre: 95249:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2485.241781] Lustre: 95249:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2485.252887] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2485.285802] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2485.607211] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2485.933770] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2485.943197] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2486.263115] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2486.268264] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2486.284086] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2486.298610] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2486.945469] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2486.954659] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2487.280767] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2487.292258] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2488.572770] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2488.578657] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2489.224846] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2489.233838] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2489.559221] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2489.566806] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2492.824208] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2492.835623] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2493.475072] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2493.484692] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2493.807249] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2493.826129] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2501.119955] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2501.126993] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 25 previous similar messages [ 2501.767940] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2501.790026] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2502.112575] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2502.124437] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2517.273501] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2517.279366] Lustre: 101492:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2517.921584] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2517.931400] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2518.247887] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2518.252714] LustreError: 101492:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2543.071263] Lustre: 101492:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2543.081084] Lustre: 101492:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2543.088728] Lustre: 101492:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 22802875-493b-4bb1-bbde-5e593ab9fe6f@192.168.203.22@tcp [ 2543.097841] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2543.102153] Lustre: 101492:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@000000003463e1dc x1877054089925376/t0(8589934779) o101->22802875-493b-4bb1-bbde-5e593ab9fe6f@192.168.203.22@tcp:62/0 lens 592/0 e 0 to 0 dl 1790099022 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2543.116359] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2543.127272] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2543.134774] Lustre: lustre-MDT0000: Denying connection for new client 22802875-493b-4bb1-bbde-5e593ab9fe6f (at 192.168.203.22@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2543.154446] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2543.158125] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2543.160307] Lustre: Skipped 2 previous similar messages [ 2543.181204] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2543.181246] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2548.704334] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2548.708711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2548.711369] Lustre: Skipped 6 previous similar messages [ 2554.651880] Lustre: server umount lustre-OST0000 complete [ 2556.897658] LustreError: 97072:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2556.909275] LustreError: 97072:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 25 previous similar messages [ 2571.231134] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2571.426567] Lustre: server umount lustre-OST0001 complete [ 2588.127162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2588.290414] Lustre: server umount lustre-MDT0000 complete [ 2590.578360] LustreError: 95231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099058 with bad export cookie 1231575946549495342 [ 2590.580840] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2590.590640] LustreError: 95231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2590.770607] Lustre: server umount lustre-MDT0001 complete [ 2595.668675] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 13:44:23 (1790099063) [ 2600.103577] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2600.113681] Lustre: *** cfs_fail_loc=197, val=0*** [ 2615.647942] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2620.769491] LustreError: 103465:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.122@tcp: failed processing log, type 4: rc = -110 [ 2625.903405] Lustre: *** cfs_fail_loc=197, val=0*** [ 2625.906886] Lustre: Skipped 65 previous similar messages [ 2649.504815] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2653.530154] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2654.716261] Lustre: server umount lustre-OST0000 complete [ 2658.934982] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 13:45:26 (1790099126) [ 2674.517605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2674.745044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2674.748576] Lustre: Skipped 2 previous similar messages [ 2676.583974] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2682.027830] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2684.369587] LustreError: 105116:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2684.375824] LustreError: 105116:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2684.381618] LustreError: 105116:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -5 [ 2684.536904] Lustre: server umount lustre-MDT0000 complete [ 2700.393323] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 2705.030797] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2722.722302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2722.837182] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2722.858602] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2722.900716] Lustre: lustre-MDT0000: new disk, initializing [ 2722.947516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2724.997852] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2731.204802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2731.278452] Lustre: 109280:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2731.425668] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2734.287545] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2742.098557] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2749.267905] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2756.776409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2757.076288] Lustre: 111254:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2763.266906] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2766.287446] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2777.555305] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2783.635261] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2783.844122] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2788.216777] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2788.419530] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2793.957479] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2793.960243] LustreError: Skipped 2 previous similar messages [ 2793.964972] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2793.978804] Lustre: Skipped 10 previous similar messages [ 2796.213917] Lustre: server umount lustre-OST0000 complete [ 2809.313091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2809.317587] Lustre: Skipped 21 previous similar messages [ 2813.407120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2813.534719] Lustre: server umount lustre-MDT0000 complete [ 2814.432713] LustreError: 109294:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2814.445027] LustreError: 109294:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 2815.733716] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099284 with bad export cookie 1231575946549505968 [ 2815.744771] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2815.750916] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2816.000469] Lustre: server umount lustre-MDT0001 complete [ 2821.845689] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 13:48:08 (1790099288) [ 2828.232274] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 2830.526765] Key type lgssc unregistered [ 2830.775465] LNet: 113741:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2830.778806] LNetError: 113741:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2830.795136] LNet: Removed LNI 192.168.203.122@tcp [ 2831.482768] Key type .llcrypt unregistered [ 2831.484676] Key type ._llcrypt unregistered [ 2845.532951] Key type ._llcrypt registered [ 2845.534454] Key type .llcrypt registered [ 2845.638268] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 2857.869780] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2858.599950] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2858.708608] alg: No test for adler32 (adler32-zlib) [ 2859.701329] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 2859.902282] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 2861.567195] Key type lgssc registered [ 2862.579138] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2893.062401] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2893.090838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2894.262985] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2894.299625] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2894.369887] Lustre: lustre-MDT0000: new disk, initializing [ 2894.433794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2894.452125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2897.393208] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2904.666775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2904.882463] Lustre: lustre-OST0000: new disk, initializing [ 2904.888530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2904.895390] Lustre: Skipped 1 previous similar message [ 2904.901410] Lustre: 117904:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2904.968730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2909.047383] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2910.765352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2910.771864] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2910.790267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2916.849511] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2916.961590] Lustre: lustre-OST0001: new disk, initializing [ 2916.965221] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2916.969894] Lustre: 118767:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2917.029656] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2918.619642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2918.637526] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2918.677723] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2921.488040] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2929.123540] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2929.135149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2929.141344] Lustre: Skipped 1 previous similar message [ 2934.241282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2934.250127] Lustre: Skipped 2 previous similar messages [ 2934.669096] Lustre: server umount lustre-MDT0000 complete [ 2939.646513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2943.345197] LustreError: 117092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099411 with bad export cookie 6878268286292892760 [ 2943.351525] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2943.363615] LustreError: 117092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2943.442928] Lustre: server umount lustre-OST0000 complete [ 2946.286523] Lustre: server umount lustre-OST0001 complete [ 2952.705926] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 13:50:19 (1790099419) [ 2969.998808] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 13:50:36 (1790099436) [ 2978.017825] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2994.924465] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 13:51:01 (1790099461) [ 3022.518698] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 3031.620951] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3060.601645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3060.790067] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3060.807505] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3060.852930] Lustre: lustre-MDT0000: new disk, initializing [ 3060.897402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3060.914620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3064.516551] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3072.913146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3072.991868] Lustre: 126516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3073.025840] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3073.029664] Lustre: Skipped 1 previous similar message [ 3073.090699] Lustre: lustre-MDT0001: new disk, initializing [ 3073.177295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3073.201919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3073.217110] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3076.466640] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3084.275098] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3088.455747] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3093.464616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3093.641187] Lustre: lustre-OST0000: new disk, initializing [ 3093.645505] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3093.650908] Lustre: 128481:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3093.729815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3098.549392] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3103.308710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3103.331819] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3103.478746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3106.357218] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3110.409886] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3110.566729] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3115.002779] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3115.188100] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3119.073382] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3119.073628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3119.093409] LustreError: Skipped 1 previous similar message [ 3119.094435] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3119.117092] Lustre: Skipped 1 previous similar message [ 3124.194787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3124.201851] Lustre: Skipped 1 previous similar message [ 3129.319552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3129.338655] Lustre: Skipped 2 previous similar messages [ 3131.359317] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3131.522605] Lustre: server umount lustre-OST0000 complete [ 3134.944632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3134.951048] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3139.553119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3139.554161] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3139.561852] Lustre: Skipped 1 previous similar message [ 3140.862528] Lustre: server umount lustre-MDT0000 complete [ 3143.992650] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099612 with bad export cookie 6878268286292893985 [ 3143.996516] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3144.001026] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3144.201465] Lustre: server umount lustre-MDT0001 complete [ 3150.697209] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 13:53:37 (1790099617) [ 3154.996522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3155.334824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3158.455307] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3160.545686] LustreError: 130893:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3160.559849] LustreError: 130893:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3164.945122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3168.519923] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3173.913673] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3177.961143] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3182.955635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3187.492304] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3192.398780] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3206.110859] Lustre: *** cfs_fail_loc=159, val=0*** [ 3206.113989] Lustre: Skipped 2 previous similar messages [ 3222.496022] Lustre: lustre-MDT0000: Client 32b20b01-cb7d-422a-8622-8b78eff7dda0 (at 192.168.203.22@tcp) reconnecting [ 3223.594419] Lustre: *** cfs_fail_loc=159, val=0*** [ 3223.596252] Lustre: Skipped 2 previous similar messages [ 3238.869847] Lustre: lustre-MDT0000: Client 32b20b01-cb7d-422a-8622-8b78eff7dda0 (at 192.168.203.22@tcp) reconnecting [ 3257.313949] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3257.321086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3257.331768] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3257.335307] Lustre: Skipped 1 previous similar message [ 3261.646909] Lustre: server umount lustre-OST0000 complete [ 3267.556184] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3267.562478] LustreError: Skipped 1 previous similar message [ 3267.567823] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3267.578125] Lustre: Skipped 1 previous similar message [ 3270.348744] Lustre: server umount lustre-MDT0000 complete [ 3272.898843] LustreError: 130873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099741 with bad export cookie 6878268286292894867 [ 3272.905049] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3272.909449] LustreError: 130873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3273.136697] Lustre: server umount lustre-MDT0001 complete [ 3279.972772] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3281.536471] Key type lgssc unregistered [ 3281.724672] LNet: 134689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3281.728695] LNetError: 134689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3281.744873] LNet: Removed LNI 192.168.203.122@tcp [ 3282.196584] Key type .llcrypt unregistered [ 3282.199046] Key type ._llcrypt unregistered [ 3296.094696] Key type ._llcrypt registered [ 3296.096216] Key type .llcrypt registered [ 3296.177184] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3296.749333] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3296.783806] alg: No test for adler32 (adler32-zlib) [ 3297.704928] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 3297.821379] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3299.431208] Key type lgssc registered [ 3300.025599] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3304.491313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3304.504234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3305.758508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3308.300729] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3313.544786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3313.812810] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3316.542215] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3321.403415] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3324.899487] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3328.843603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3329.063853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3331.113991] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3332.528973] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3336.896979] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3347.329479] Lustre: *** cfs_fail_loc=159, val=0*** [ 3363.787043] Lustre: lustre-MDT0000: Client b70e73f5-fa13-427f-865d-fc2fd53d9b95 (at 192.168.203.22@tcp) reconnecting [ 3365.354806] Lustre: *** cfs_fail_loc=159, val=0*** [ 3365.358416] Lustre: Skipped 1 previous similar message [ 3381.709844] Lustre: lustre-MDT0001: Client b70e73f5-fa13-427f-865d-fc2fd53d9b95 (at 192.168.203.22@tcp) reconnecting [ 3382.532337] Lustre: *** cfs_fail_loc=159, val=0*** [ 3382.539187] Lustre: Skipped 2 previous similar messages [ 3398.105045] Lustre: lustre-MDT0001: Client b70e73f5-fa13-427f-865d-fc2fd53d9b95 (at 192.168.203.22@tcp) reconnecting [ 3414.048573] Lustre: 139157:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3416.084052] Lustre: *** cfs_fail_loc=159, val=0*** [ 3416.087144] Lustre: Skipped 2 previous similar messages [ 3432.413875] Lustre: lustre-MDT0000: Client 88b0cf98-1d1a-42a8-b40f-be698b962ab0 (at 192.168.203.22@tcp) reconnecting [ 3433.360458] Lustre: *** cfs_fail_loc=159, val=0*** [ 3433.365019] Lustre: Skipped 15 previous similar messages [ 3449.818847] Lustre: lustre-MDT0000: Client 88b0cf98-1d1a-42a8-b40f-be698b962ab0 (at 192.168.203.22@tcp) reconnecting [ 3450.904725] Lustre: 139446:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3469.280811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3469.284320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3469.290904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3472.353937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3472.357489] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3472.365205] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3473.870328] Lustre: server umount lustre-OST0000 complete [ 3477.473490] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3477.475814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3477.482705] Lustre: Skipped 1 previous similar message [ 3477.489799] Lustre: Skipped 2 previous similar messages [ 3482.484281] Lustre: server umount lustre-MDT0000 complete [ 3482.593083] LustreError: 136091:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3482.602407] LustreError: 136091:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3484.546085] LustreError: 136077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790099952 with bad export cookie 2279989133075776322 [ 3484.547325] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3484.553146] LustreError: 136077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3484.713825] Lustre: server umount lustre-MDT0001 complete [ 3490.282354] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3491.782046] Key type lgssc unregistered [ 3491.951859] LNet: 140521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3491.958832] LNetError: 140521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3491.973305] LNet: Removed LNI 192.168.203.122@tcp [ 3492.430804] Key type .llcrypt unregistered [ 3492.432256] Key type ._llcrypt unregistered [ 3506.836636] Key type ._llcrypt registered [ 3506.838748] Key type .llcrypt registered [ 3506.902559] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3507.370871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3507.413110] alg: No test for adler32 (adler32-zlib) [ 3508.326567] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 3508.484361] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3510.095197] Key type lgssc registered [ 3510.721811] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3514.939565] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3514.953334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3516.195094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3518.347538] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3523.034867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3523.263362] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3525.593132] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3529.614664] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3532.656961] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3536.232869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3536.414353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3539.389866] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3541.543929] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3543.124669] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3548.716474] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3550.847676] Lustre: 144505:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3552.506557] Lustre: 144553:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3553.451492] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3554.014460] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3555.913306] Lustre: 144700:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3556.420345] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3556.880992] Lustre: 144798:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3561.953804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3561.962600] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3561.971809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3564.001531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3564.004395] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3564.005408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3565.056536] Lustre: server umount lustre-OST0000 complete [ 3569.120412] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3569.131904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3569.136024] Lustre: Skipped 2 previous similar messages [ 3573.379166] Lustre: server umount lustre-MDT0000 complete [ 3574.242774] LustreError: 141928:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3574.250073] LustreError: 141928:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3575.266518] LustreError: 141908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790100043 with bad export cookie 17542882674450291434 [ 3575.271806] LustreError: 141908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3575.273631] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3575.439473] Lustre: server umount lustre-MDT0001 complete [ 3581.101148] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3582.665974] Key type lgssc unregistered [ 3582.816452] LNet: 145854:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3582.820687] LNetError: 145854:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3582.835928] LNet: Removed LNI 192.168.203.122@tcp [ 3583.247388] Key type .llcrypt unregistered [ 3583.249035] Key type ._llcrypt unregistered [ 3596.085795] Key type ._llcrypt registered [ 3596.088951] Key type .llcrypt registered [ 3596.156196] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3596.683966] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3596.693922] alg: No test for adler32 (adler32-zlib) [ 3597.592990] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 3597.723873] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3599.328436] Key type lgssc registered [ 3599.955257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3604.076430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3604.092269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3605.369423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3607.878120] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3612.739019] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3612.928923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3615.462623] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3621.687782] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3628.186240] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3638.722921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3639.189992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3647.241515] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3650.502707] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3656.362322] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3658.745727] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3663.922382] Lustre: *** cfs_fail_loc=159, val=0*** [ 3663.929298] Lustre: Skipped 2 previous similar messages [ 3680.262071] Lustre: lustre-MDT0000: Client 7a417e44-bab8-45da-9527-a140cd3d44ad (at 192.168.203.22@tcp) reconnecting [ 3685.868158] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3685.880075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3685.899112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3687.493899] Lustre: server umount lustre-OST0000 complete [ 3694.570574] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3694.574248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3694.591297] Lustre: Skipped 1 previous similar message [ 3694.615257] Lustre: Skipped 1 previous similar message [ 3697.054419] Lustre: server umount lustre-MDT0000 complete [ 3699.681993] LustreError: 147272:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3699.723148] LustreError: 147272:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3700.954713] LustreError: 149124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790100169 with bad export cookie 974343598472858856 [ 3700.957330] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3700.966926] LustreError: 149124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3711.704839] Lustre: server umount lustre-MDT0001 complete [ 3721.908950] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3724.297951] Key type lgssc unregistered [ 3724.545589] LNet: 150937:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3724.552507] LNetError: 150937:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3724.568638] LNet: Removed LNI 192.168.203.122@tcp [ 3725.272406] Key type .llcrypt unregistered [ 3725.274385] Key type ._llcrypt unregistered [ 3743.684954] Key type ._llcrypt registered [ 3743.686556] Key type .llcrypt registered [ 3743.787618] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3744.344661] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3744.393748] alg: No test for adler32 (adler32-zlib) [ 3745.342845] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 3745.495188] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3747.095200] Key type lgssc registered [ 3747.958443] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3753.641125] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3753.652406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3754.953671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3758.180092] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3764.112995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3764.366061] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3767.874650] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3773.737716] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3778.312677] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3783.243878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3783.496029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3787.598618] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3789.675597] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3792.896641] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3799.931902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3820.523783] Lustre: lustre-OST0000: evicting 48da730c-e7bc-46c9-95ce-f8976811c318 (at 192.168.203.22@tcp) by administrative request [ 3820.530533] Lustre: Skipped 1 previous similar message [ 3851.233425] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3851.240711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3851.246199] Lustre: Skipped 1 previous similar message [ 3851.254291] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3853.011355] Lustre: server umount lustre-OST0000 complete [ 3856.353415] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3856.355305] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3856.365387] LustreError: Skipped 1 previous similar message [ 3856.366663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3856.378777] Lustre: Skipped 1 previous similar message [ 3856.400976] Lustre: Skipped 2 previous similar messages [ 3861.473170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3861.478968] Lustre: Skipped 1 previous similar message [ 3861.719882] Lustre: server umount lustre-MDT0000 complete [ 3864.174904] LustreError: 152327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790100332 with bad export cookie 14015045916814618401 [ 3864.180811] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3864.185509] LustreError: 152327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3864.420980] Lustre: server umount lustre-MDT0001 complete [ 3872.296975] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3874.341596] Key type lgssc unregistered [ 3874.575346] LNet: 156436:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3874.581736] LNetError: 156436:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3874.597390] LNet: Removed LNI 192.168.203.122@tcp [ 3875.199138] Key type .llcrypt unregistered [ 3875.202743] Key type ._llcrypt unregistered [ 3931.534844] Key type ._llcrypt registered [ 3931.536523] Key type .llcrypt registered [ 3931.601685] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3932.254423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3932.300417] alg: No test for adler32 (adler32-zlib) [ 3933.409481] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 3933.595951] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3935.215330] Key type lgssc registered [ 3936.563442] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3944.142809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3944.162275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3945.522627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3949.106487] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3955.691660] LustreError: 158918:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3955.707705] LustreError: 158918:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3957.519576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3957.904733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3961.222824] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3968.348086] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3975.953485] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3987.344669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3987.879374] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3989.947430] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3995.528432] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4003.274973] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4009.241918] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4025.824838] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4025.838852] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4025.888986] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4026.292238] Lustre: server umount lustre-OST0000 complete [ 4034.017839] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4034.021164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4034.046627] Lustre: Skipped 1 previous similar message [ 4034.058537] Lustre: Skipped 1 previous similar message [ 4036.411780] Lustre: server umount lustre-MDT0000 complete [ 4039.151736] LustreError: 158918:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4040.549056] LustreError: 158898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790100508 with bad export cookie 17800941124225490647 [ 4040.559495] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4040.575608] LustreError: 158898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4050.087469] Lustre: server umount lustre-MDT0001 complete [ 4065.292116] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4068.951594] Key type lgssc unregistered [ 4069.290711] LNet: 162494:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4069.299316] LNetError: 162494:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4069.315697] LNet: Removed LNI 192.168.203.122@tcp [ 4070.397393] Key type .llcrypt unregistered [ 4070.401625] Key type ._llcrypt unregistered [ 4098.383298] Key type ._llcrypt registered [ 4098.385895] Key type .llcrypt registered [ 4098.483987] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4099.507974] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4099.631708] alg: No test for adler32 (adler32-zlib) [ 4100.812308] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 4101.025692] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4102.719321] Key type lgssc registered [ 4103.929332] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4120.409272] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:09:47 (1790100587) [ 4136.734789] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 4147.573197] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4185.952975] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4185.987706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4187.213940] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4187.240968] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4187.313199] Lustre: lustre-MDT0000: new disk, initializing [ 4187.401231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4187.422126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4191.729606] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4202.221595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4202.312500] Lustre: 166773:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4202.366490] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4202.370416] Lustre: Skipped 1 previous similar message [ 4202.422894] Lustre: lustre-MDT0001: new disk, initializing [ 4202.550517] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4202.604727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4202.611762] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4207.171359] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4216.902897] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4223.377700] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4231.052865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4231.324353] Lustre: lustre-OST0000: new disk, initializing [ 4231.329115] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4231.343353] Lustre: 168744:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4231.482304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4237.715662] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4239.399906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4239.410218] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4239.534248] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4247.452225] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4260.328806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4260.331236] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4260.344114] LustreError: Skipped 1 previous similar message [ 4260.347117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4260.391040] Lustre: Skipped 1 previous similar message [ 4265.444132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4265.452029] Lustre: Skipped 2 previous similar messages [ 4265.970771] Lustre: server umount lustre-OST0000 complete [ 4270.561723] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4270.572620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4270.584404] Lustre: Skipped 1 previous similar message [ 4270.602957] Lustre: Skipped 1 previous similar message [ 4275.681733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4275.692517] Lustre: Skipped 1 previous similar message [ 4276.460592] Lustre: server umount lustre-MDT0000 complete [ 4280.124726] LustreError: 167568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790100748 with bad export cookie 7565642894891687390 [ 4280.126504] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4280.134208] LustreError: 167568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4280.483692] Lustre: server umount lustre-MDT0001 complete [ 4292.859532] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4295.854722] Key type lgssc unregistered [ 4296.088263] LNet: 170648:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4296.094406] LNetError: 170648:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4296.108615] LNet: Removed LNI 192.168.203.122@tcp [ 4297.037606] Key type .llcrypt unregistered [ 4297.042503] Key type ._llcrypt unregistered [ 4321.905261] Key type ._llcrypt registered [ 4321.907110] Key type .llcrypt registered [ 4321.978742] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4322.872353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4322.886455] alg: No test for adler32 (adler32-zlib) [ 4323.889562] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 4324.059469] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4325.775219] Key type lgssc registered [ 4327.301166] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4336.304210] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4336.360222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4337.941056] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4342.733707] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4348.386034] LustreError: 172080:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4348.408914] LustreError: 172080:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4351.483855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4351.871439] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4356.574791] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4364.715898] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4371.074308] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4377.633746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4377.959059] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4383.090918] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4383.617517] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4392.094365] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4402.637661] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4402.657705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4402.673788] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4402.694712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.718502] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4404.078340] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4404.106656] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4404.125910] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4404.134734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.157772] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4405.378125] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4405.411626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4405.422601] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4405.432374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.446605] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4406.437184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.594660] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4407.601973] Lustre: Skipped 1 previous similar message [ 4407.620443] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4407.632104] Lustre: Skipped 1 previous similar message [ 4407.638593] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4407.653549] Lustre: Skipped 1 previous similar message [ 4407.661914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.678663] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4407.692558] Lustre: Skipped 1 previous similar message [ 4408.700489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.818942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.843740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.621237] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4411.628919] Lustre: Skipped 3 previous similar messages [ 4411.649107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4411.659272] Lustre: Skipped 3 previous similar messages [ 4411.662817] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4411.666964] Lustre: Skipped 3 previous similar messages [ 4411.669930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.678603] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4411.684175] Lustre: Skipped 3 previous similar messages [ 4412.708266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.676083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.725193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.670387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.724338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.682422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.591704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.453169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.349569] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4420.357195] Lustre: Skipped 8 previous similar messages [ 4420.369724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4420.378699] Lustre: Skipped 8 previous similar messages [ 4420.382721] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4420.386156] Lustre: Skipped 8 previous similar messages [ 4420.389932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.398130] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4420.403692] Lustre: Skipped 8 previous similar messages [ 4421.231938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.140897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.063245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.228822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.188691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.937161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.714132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.499455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.506486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.554629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.660517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.528375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.564587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.519905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.546656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.528688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.384752] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4436.389301] Lustre: Skipped 16 previous similar messages [ 4436.413637] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4436.425856] Lustre: Skipped 16 previous similar messages [ 4436.430510] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4436.435097] Lustre: Skipped 16 previous similar messages [ 4436.439924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.450306] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4436.453974] Lustre: Skipped 16 previous similar messages [ 4437.352554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.443899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.414769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.519752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.486943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.545382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.394894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.489724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.310555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.434650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.441470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.431585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.521209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.623460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.709971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.755470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.616419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.563612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.650162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.464214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.444633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.397372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.423976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.298509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.322362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.344638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.250695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.108391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.986642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.826943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.760828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.620732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.475709] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4468.479943] Lustre: Skipped 32 previous similar messages [ 4468.494512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4468.501453] Lustre: Skipped 32 previous similar messages [ 4468.505140] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4468.512489] Lustre: Skipped 32 previous similar messages [ 4468.515520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.523403] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4468.529059] Lustre: Skipped 32 previous similar messages [ 4469.432364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.333583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.287342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.208728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.145522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.939091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.952326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.842159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.919877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.930628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.876151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.650360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.706526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.753583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.796130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.870525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.927593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.730322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.632661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.729958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.673188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.782610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.715142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.864090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.049293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.195477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.367649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.417506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.504434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.575611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.681240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.542596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.577566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.625180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.748668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.768155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.909315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.031282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.244921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.440934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.613790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.509618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.492275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.695112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.693723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.820302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.801500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.826753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.937400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.767530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.861474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.889194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.022580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.198929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.295799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.536392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.572441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.534888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.767444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.141858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.548733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4532.554021] Lustre: Skipped 60 previous similar messages [ 4532.597619] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.616600] Lustre: Skipped 60 previous similar messages [ 4532.620296] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4532.627182] Lustre: Skipped 60 previous similar messages [ 4532.632472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.641126] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4532.653151] Lustre: Skipped 60 previous similar messages [ 4533.557965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.606536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.606578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.837919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.878816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.856301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.902371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.857369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.910847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.889489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.096638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.139503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.640535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.946621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.058989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.029633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.932288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.854088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.157163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.209829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.133922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.130720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.943574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.073888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.930159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.860719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.970940] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4560.987101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.994376] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4561.738275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.575558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.336485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.154490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.932462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.879772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.756363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.847740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.524068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.359433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.221590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.211039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.477237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.634074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.389760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.973960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.640257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.555956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.669769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.692121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.717613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.517213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.447562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.295517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.122636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.053479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.049863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.267260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.518815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.951727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.697509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.904731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.357778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.387466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.534687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.571353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.583825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.591116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.794730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.891071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.846088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.948462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.999738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.897660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.940814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.995341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.084814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.081363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.214190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.152648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.092387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.018120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.038817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.913424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.029141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.921460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.083777] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4617.089982] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4617.099915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.100300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.105848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.057785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.209823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.133704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.192508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.283897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.307648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.332982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.219982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.223565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.234569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.251050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.367771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.485136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.449975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.587246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.614335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.616328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.515663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.600576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.440538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.460783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.403954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.412772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.272385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.180802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.175042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.127246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.134876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.210683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.114926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.125033] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4650.140284] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4650.143119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.195564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.144195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.060726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.924643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.823476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.919245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.779474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.767872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.762664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.713600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.592873] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4660.598300] Lustre: Skipped 127 previous similar messages [ 4660.615244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4660.638797] Lustre: Skipped 127 previous similar messages [ 4660.641847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4660.648161] Lustre: Skipped 127 previous similar messages [ 4660.652104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.663560] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4660.670078] Lustre: Skipped 127 previous similar messages [ 4661.692277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.539061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.463486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.463566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.512603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.629765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.751883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.764620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.842650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.983600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.956629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.149347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.365450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.338219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.562224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.702304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.833705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.795820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.765075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.705602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.585993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.359320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.310296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.172194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.965788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.856259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.790413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.654581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.590909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.381781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.497625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.179670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.015264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.915954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.145244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.445856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.589641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.709841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.818132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.063456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.164918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.203810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.357810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.548959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.809533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.701385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.686382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.959387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.911135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.128550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.279048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.157777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.303686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.488185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.480315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.656901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.688527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.607361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.694200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.873559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.059299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.959026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.068636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.062799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.199132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.415658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.660486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.623264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.652693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.675802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.789195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.761651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.790266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.759189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.787774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.790477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.790794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.884389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.879164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.866404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.875831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.093317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.113588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.016479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.070378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.066836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.894914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.962688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.435581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.301240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.117407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.012638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.981695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.821178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.689342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.663193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.737340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.706360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.710330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.642569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.990095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.530358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.736528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.944414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.381198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.551713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.626244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.546891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.424935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.268211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.381918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.359582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.400768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.326714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.310217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.224574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.219629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.247674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.125710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.046071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.910764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.732444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.881420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.898437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.892771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.997303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.997388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.957700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.944205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.827529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.854196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.793341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.779514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.748141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.787369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.764604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.868902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.934246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.925988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.916490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.800225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.763651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.777256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.804344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.017993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.113867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.118721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.334681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.545867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.687046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.553949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.568200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.582660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.773053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.966924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.168564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.152905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.131627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.105887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.270980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.303662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.201034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.310501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.419692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.280187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.116813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.927937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.856807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.776203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.728571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.602358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.545792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.460706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.591882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.791348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.709527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.835124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.965633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.019340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.951763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.907840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.826263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.917603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.935757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.027624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.153544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.261558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.130917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.159568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.030439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.085132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.022641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.985772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.973084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.969368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.821415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.760729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.627395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.519958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.419266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.295448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.138726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.021735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.816077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.706464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.720600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.794524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.785686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.773367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.794214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.901167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.914079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.974044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.152453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.087235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.123607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.952791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.863418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.641137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.442042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.197152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.963884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.776032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.659076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.461492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.359250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.207531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.149214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.186933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.119713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.196131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.110486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.904396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.780120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.726055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.493807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.493367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.324266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.357156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.183477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.132765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.042778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.053701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.012211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.996251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.902643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.818960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.847432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.844382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.811597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.646693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.644803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.608607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.507620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.494620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.337884] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4917.342779] Lustre: Skipped 255 previous similar messages [ 4917.356344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4917.363159] Lustre: Skipped 255 previous similar messages [ 4917.366829] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4917.371339] Lustre: Skipped 255 previous similar messages [ 4917.374775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.382893] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4917.388416] Lustre: Skipped 255 previous similar messages [ 4918.352158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.161422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.974617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.887072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.809984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.748099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.617934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.970564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.937464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.841675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.752890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.593349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.614987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.653774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.588232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.576440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.471825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.569493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.499307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.635748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.754257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.789986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.876878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.874222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.888419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.906169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.064738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.069175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.013564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.150223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.079135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.080904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.066513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.975379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.948152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.915425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.790106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.677540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.744251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.656968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.764132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.857958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.862850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.997325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.054405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.975343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.880066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.985872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.075726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.991703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.957528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.889224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.771829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.756775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.862258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.753853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.823679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.926535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.076311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.137670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.177793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.144649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.215847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.169586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.058362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.051329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.104088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.147216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.039630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.949350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.113061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.402475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.416432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.385612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.307641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.342904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.316051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.331677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.392949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.373115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.330380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.305322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.523404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.435598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.458297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.424055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.445620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.595642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.472514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.357913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.180542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.176551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.086271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.076612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.995574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.828902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.614842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.491610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.282645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.315571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.274819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.097967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.078991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.049471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.929689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.864289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.809256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.826182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.859890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.994528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.964781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.956770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.873673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.938508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.025869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.332214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.432430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.457187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.446654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.434689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.581255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.587846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.582426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.722580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.705212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.924060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.029647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.169644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.077357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.967396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.847236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.842611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.796975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.843717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.775362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.730498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.652698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.656233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.575512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.593703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.732654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.587864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.604552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.544783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.604836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.498945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.550603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.558608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.460704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.273960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.201278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.031752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.968988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.869418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.867418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.889601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.977888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.987413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.877704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.608583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.490359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.459597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.412772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.446717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.457168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.498177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.351899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.390256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.416666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.269411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.296618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.254870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.230863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.095400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.072408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.994982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.957110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.904271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.842580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.874361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.842101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.735465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.633433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.476411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.303529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.202688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.087723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.235650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.009889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.905222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.666845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.548115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.337037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.216812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.028756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.895961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.707506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.556364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.322054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.173029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.066401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.029772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.919927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.920103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.747289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.634109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.333189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.295054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.057691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.837026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.710004] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.727899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.692917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.734339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.612038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.431439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.276149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.159320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.030236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.884396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.710912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.655549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.473738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.587385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.557560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.605625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.638708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.560364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.367745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.340634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.244718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.294848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.403290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.294493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.085144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.838842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.639502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.508250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.342049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.184037] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5148.192293] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5148.203718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.921247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.654396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.427523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.288035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.200764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.003872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.699602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.645397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.462927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.323604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.161766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.216749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.023721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.920702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.816108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.906605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.737199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.714493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.554842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.516190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.410210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.300406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.069044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.849748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.530258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.285950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.041874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.052914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.829814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.712042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.641872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.811855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.732022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.716495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.607943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.498823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.435984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.327163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.117852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.009649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.894402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.724587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.661046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.410736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.237337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.150779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.064779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.940252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.808420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.849656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.663609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.521708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.404530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.172224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.042810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.787632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.570272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.378313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.085397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.864707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.829152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.874752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.886423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.662290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.671767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.453065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.346536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.137945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.031159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.798738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.657888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.656515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.581742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.523278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.371895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.164978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.017212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.754704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.524074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.874587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.717177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.597733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.573354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.533080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.373468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.426505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.556864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.611339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.516304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.455632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.522040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.392985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.381974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.378660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.810037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.833940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.923905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.010305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.122836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.520111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.409042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.500509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.659875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.965926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.253892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.564647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.495272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.489078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.221268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.347718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.442138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.448316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.293341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.037823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.139954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.442495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.200126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.186784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.963374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.748394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.606988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.782201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.011582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.185210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.911124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.883252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.902678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.833793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.596835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.532484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.640744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.623270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.063627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.062348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.181052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.986471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.065421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.216180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.247205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.379090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.676389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.215229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.650096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.043970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.194170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.197977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.101516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.457154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.980915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.081247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.072506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.881401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.653502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.656863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.733259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.414438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.164353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.885392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.551905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.216213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.072978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.809818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.634976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.424711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.260401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.898734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.315714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.270868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.149512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.772906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.966796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.194731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.750899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.116705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.128661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.940893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.921451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.147915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.006404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.005831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.831468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.760831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.881279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.751392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.718369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.642781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.604254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.611539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.721552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.755894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.817368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.833873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.927673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.619431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.641276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.578569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.623450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.568841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.476137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.323535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.225973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.157503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.092218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.931112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.932812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.998167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.049763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.176772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.110081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.125635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.199554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.083471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.086376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.914519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.032282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.954535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.942924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.889780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.859497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.893951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.987648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.047929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.081905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.011974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.887597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.830913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.824324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.890532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.893650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.891663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.869134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.796661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.747530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.800944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.777227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.666651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.730422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.608750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.687221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.619555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.533511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.454492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.284984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.088594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.049504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.988455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.834070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.472058] LustreError: 173903:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5802.483122] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6530.539830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6530.542527] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6530.547493] Lustre: Skipped 488 previous similar messages [ 6530.559034] Lustre: Skipped 1 previous similar message [ 6535.694467] LustreError: 226128:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6535.719907] LustreError: 226128:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6535.932935] Lustre: server umount lustre-OST0000 complete [ 6540.777392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6540.789709] Lustre: Skipped 1 previous similar message [ 6545.893969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6545.900715] Lustre: Skipped 1 previous similar message [ 6546.463829] Lustre: server umount lustre-MDT0000 complete [ 6550.062283] LustreError: 172062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103018 with bad export cookie 16810260481406079233 [ 6550.070736] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6550.084642] LustreError: 172062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6550.730957] Lustre: server umount lustre-MDT0001 complete [ 6566.063323] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 6570.435303] Key type lgssc unregistered [ 6570.899665] LNet: 227356:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6570.906260] LNetError: 227356:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6570.926607] LNet: Removed LNI 192.168.203.122@tcp [ 6571.978811] Key type .llcrypt unregistered [ 6571.981759] Key type ._llcrypt unregistered [ 6603.660301] Key type ._llcrypt registered [ 6603.663915] Key type .llcrypt registered [ 6603.845605] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 6604.695409] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6604.784749] alg: No test for adler32 (adler32-zlib) [ 6605.843231] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 6606.018649] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 6607.792341] Key type lgssc registered [ 6609.608277] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6618.896444] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6618.928913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6620.548394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6624.970958] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6629.872327] LustreError: 228790:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6629.906627] LustreError: 228790:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6633.665732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6634.026257] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6638.043299] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6645.138616] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6651.030981] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6657.864618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6658.451472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6665.568924] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6665.703520] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6674.771939] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6681.277232] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6683.877758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6683.897255] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6683.903744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.929755] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6683.931205] LustreError: 228810:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6687.182523] Lustre: lustre-OST0000: Not available for connect from 192.168.203.22@tcp (stopping) [ 6690.274440] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6690.288179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6690.319479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6691.309478] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6691.343133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6691.352360] Lustre: Skipped 1 previous similar message [ 6693.218566] Lustre: server umount lustre-OST0000 complete [ 6703.991331] LustreError: 228810:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6728.374660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6728.801536] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6730.865489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.918132] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6735.587260] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6741.164886] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.180854] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6743.948383] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6745.830255] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6751.691186] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c1b86c3d000.ost_server_uuid 50 [ 6753.312651] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c1b86c3d000.ost_server_uuid in FULL state after 0 sec [ 6758.880728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6758.901137] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6758.925139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6758.935768] Lustre: Skipped 1 previous similar message [ 6760.584139] Lustre: server umount lustre-OST0000 complete [ 6767.074169] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6767.074567] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6767.102591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6767.115852] Lustre: Skipped 1 previous similar message [ 6770.835156] Lustre: server umount lustre-MDT0000 complete [ 6772.196873] LustreError: 228787:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6772.221443] LustreError: 228787:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6774.450244] LustreError: 233244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103242 with bad export cookie 11976275388282350145 [ 6774.459793] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6774.470770] LustreError: 233244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6784.120020] Lustre: server umount lustre-MDT0001 complete [ 6798.598421] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 6801.721188] Key type lgssc unregistered [ 6801.990050] LNet: 233996:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6801.996722] LNetError: 233996:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6802.010344] LNet: Removed LNI 192.168.203.122@tcp [ 6802.929766] Key type .llcrypt unregistered [ 6802.933237] Key type ._llcrypt unregistered [ 6839.246723] Key type ._llcrypt registered [ 6839.248339] Key type .llcrypt registered [ 6839.360749] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 6840.128885] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6840.146107] alg: No test for adler32 (adler32-zlib) [ 6841.099418] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 6841.324699] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 6842.951197] Key type lgssc registered [ 6844.000532] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6854.515072] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6854.527708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6856.177831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6861.731950] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6866.401436] LustreError: 235742:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6866.416745] LustreError: 235742:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6871.276562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6871.520276] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 6871.632359] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6875.923993] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6879.163656] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6886.631905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6887.027309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6894.657507] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6901.399788] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6904.292518] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6904.576594] Lustre: lustre-OST0001: new disk, initializing [ 6904.581961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6904.593943] Lustre: 238052:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6904.701848] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6911.616374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6911.626572] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6911.693158] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 6911.762518] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6923.702737] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6931.384149] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6944.557301] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 6944.568510] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6944.598442] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6947.811316] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6947.815412] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6947.833678] Lustre: Skipped 1 previous similar message [ 6947.855587] Lustre: Skipped 3 previous similar messages [ 6950.944528] Lustre: server umount lustre-MDT0000 complete [ 6952.945872] LustreError: 235738:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6952.986757] LustreError: 235738:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 6955.841917] LustreError: 235721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103424 with bad export cookie 519518204514411515 [ 6955.843864] LustreError: 236478:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 6955.857855] LustreError: 235721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6955.859131] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6956.647494] Lustre: server umount lustre-MDT0001 complete [ 6972.538730] Lustre: server umount lustre-OST0000 complete [ 6974.306135] Lustre: 234991:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790103426/real 1790103426] req@ffff9f3104d48700 x1877059367434240/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790103442 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6974.351882] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6974.374364] Lustre: Skipped 1 previous similar message [ 6976.739794] Lustre: 234994:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790103428/real 1790103428] req@ffff9f310ad6f800 x1877059367434496/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790103444 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6978.790202] Lustre: 234991:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790103431/real 1790103431] req@ffff9f312855ed80 x1877059367434752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790103447 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6980.403743] Lustre: server umount lustre-OST0001 complete [ 7014.927491] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7026.868524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7027.258567] LustreError: 241162:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7027.361082] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7032.801806] LustreError: 241163:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7032.887651] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7036.907231] LustreError: 241162:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7041.933943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7042.016135] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7042.237107] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7047.233242] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7050.117231] Lustre: 242303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7056.966986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7064.389535] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7067.628369] LustreError: 242655:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7067.647020] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7071.748395] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7073.395409] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7073.624706] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7073.629364] Lustre: Skipped 1 previous similar message [ 7080.511600] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7088.185517] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7091.970928] Lustre: 244172:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7103.984876] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7104.004261] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7104.041551] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7110.375788] Lustre: server umount mylustre-MDT0000 complete [ 7114.646077] LustreError: 241137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103582 with bad export cookie 519518204514414378 [ 7114.650466] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7114.656955] LustreError: 241137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7114.727108] LustreError: 241159:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7114.731054] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7114.741833] LustreError: 241159:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7114.751407] Lustre: Skipped 3 previous similar messages [ 7114.768065] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7114.772553] Lustre: Skipped 9 previous similar messages [ 7115.007186] Lustre: server umount mylustre-MDT0001 complete [ 7120.794594] Lustre: server umount mylustre-OST0000 complete [ 7124.986318] Lustre: server umount mylustre-OST0001 complete [ 7145.964975] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7154.677931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7155.054602] LustreError: 246481:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7155.127369] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7159.263208] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7167.157635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7171.627925] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7174.363447] Lustre: 247623:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7180.464852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7186.730739] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7192.045380] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7194.488336] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7199.756329] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7200.729326] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7200.840854] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7207.353566] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7210.765737] Lustre: 249508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7218.657739] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7218.664156] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7218.673203] Lustre: Skipped 1 previous similar message [ 7218.677697] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7222.857974] Lustre: server umount tfs-MDT0000 complete [ 7225.571983] LustreError: 249510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103693 with bad export cookie 519518204514416331 [ 7225.579573] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7225.587228] LustreError: 249510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7225.928746] Lustre: server umount tfs-MDT0001 complete [ 7238.819162] Lustre: server umount tfs-OST0000 complete [ 7241.823093] Lustre: 234994:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790103694/real 1790103694] req@ffff9f310a6a9f80 x1877059367545088/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790103710 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7241.870171] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7241.887810] Lustre: Skipped 3 previous similar messages [ 7242.350706] Lustre: server umount tfs-OST0001 complete [ 7262.335727] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7270.816448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7271.179523] LustreError: 251761:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7271.195374] LustreError: 251761:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7271.245148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7271.249573] Lustre: Skipped 3 previous similar messages [ 7275.348885] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7282.009897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7285.804846] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7288.309593] Lustre: 252901:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7293.866327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7296.179666] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7299.084852] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7306.768955] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 7307.639309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7313.403946] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7314.220597] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7321.687509] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7325.216365] Lustre: 254772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7331.284165] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 15:03:18 (1790103798) [ 7349.221591] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7349.223246] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7349.229605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7349.229613] Lustre: Skipped 4 previous similar messages [ 7349.237906] Lustre: Skipped 1 previous similar message [ 7352.441324] Lustre: server umount lustre-MDT0000 complete [ 7355.497728] LustreError: 251738:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103823 with bad export cookie 519518204514418249 [ 7355.511696] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7355.520205] LustreError: 251738:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7361.862830] Lustre: server umount lustre-MDT0001 complete [ 7371.105868] Lustre: server umount lustre-OST0000 complete [ 7380.138340] Lustre: server umount lustre-OST0001 complete [ 7385.331551] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 7392.878624] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7418.939168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7419.131472] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7419.166811] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7419.231935] Lustre: lustre-MDT0000: new disk, initializing [ 7419.310598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7419.317640] Lustre: Skipped 3 previous similar messages [ 7419.329764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7422.845188] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7432.169730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7432.294890] Lustre: 257951:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7432.355116] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7432.359577] Lustre: Skipped 1 previous similar message [ 7432.447440] Lustre: lustre-MDT0001: new disk, initializing [ 7432.577941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7432.593399] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7436.089200] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7444.226584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7444.444437] Lustre: lustre-OST0000: new disk, initializing [ 7444.447606] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7444.455991] Lustre: 259536:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7446.298233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7446.307275] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7446.407675] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7449.786604] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7458.358956] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7458.481582] Lustre: lustre-OST0001: new disk, initializing [ 7458.484856] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7458.489745] Lustre: 260407:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7460.274348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7460.284652] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7460.356454] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7463.564251] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7473.588797] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 15:05:40 (1790103940) [ 7484.434887] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 15:05:51 (1790103951) [ 7501.287345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7501.297395] Lustre: Skipped 2 previous similar messages [ 7501.302805] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7501.307276] Lustre: Skipped 8 previous similar messages [ 7504.148281] Lustre: server umount lustre-OST0000 complete [ 7506.403612] LustreError: 259520:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7506.435769] LustreError: 259520:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 15 previous similar messages [ 7516.672158] Lustre: server umount lustre-MDT0000 complete [ 7521.973238] LustreError: 258748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790103990 with bad export cookie 519518204514420027 [ 7521.975021] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7521.999177] LustreError: 258748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7528.637520] Lustre: server umount lustre-MDT0001 complete [ 7544.481345] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 7624.604430] Lustre: server umount lustre-OST0001 complete [ 7631.416300] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 7645.376896] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7682.656087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7682.997669] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7683.024980] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7683.122044] Lustre: lustre-MDT0000: new disk, initializing [ 7683.214331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7683.221243] Lustre: Skipped 3 previous similar messages [ 7683.244820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7688.461053] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7699.869738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7700.032649] Lustre: 265264:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7700.105405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7700.111804] Lustre: Skipped 1 previous similar message [ 7700.243319] Lustre: lustre-MDT0001: new disk, initializing [ 7700.470586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7700.485963] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7705.236983] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7715.252837] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7721.235542] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7727.929522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7728.165127] Lustre: 267229:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7729.316915] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7729.506901] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7734.221443] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7743.337190] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7755.232912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7755.233178] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7755.244667] LustreError: Skipped 1 previous similar message [ 7755.245612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7755.245616] Lustre: Skipped 4 previous similar messages [ 7755.271026] Lustre: Skipped 6 previous similar messages [ 7756.499105] Lustre: server umount lustre-OST0000 complete [ 7766.580369] Lustre: server umount lustre-MDT0000 complete [ 7770.151798] LustreError: 267228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790104238 with bad export cookie 519518204514421497 [ 7770.156145] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7770.162454] LustreError: 267228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7770.503531] Lustre: server umount lustre-MDT0001 complete [ 7781.827588] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 7784.511249] Key type lgssc unregistered [ 7784.793671] LNet: 268989:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7784.798765] LNetError: 268989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7784.822878] LNet: Removed LNI 192.168.203.122@tcp [ 7785.683513] Key type .llcrypt unregistered [ 7785.685359] Key type ._llcrypt unregistered [ 7814.650301] Key type ._llcrypt registered [ 7814.652272] Key type .llcrypt registered [ 7814.757910] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7816.287781] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7816.420246] alg: No test for adler32 (adler32-zlib) [ 7817.559298] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 7817.845246] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 7819.543233] Key type lgssc registered [ 7820.889940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7829.574480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7829.595631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7831.087929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7836.340486] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7841.252382] LustreError: 270468:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7841.273565] LustreError: 270468:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7846.052184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7846.407789] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7850.985104] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7858.192832] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7864.230772] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7872.118991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7872.428135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7876.534222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7879.355348] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7887.891845] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7889.068765] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7891.949360] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7891.956542] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7891.981901] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7896.033354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7896.038494] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7896.040635] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7896.050432] Lustre: Skipped 1 previous similar message [ 7901.153887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7905.247319] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7905.443592] Lustre: server umount lustre-OST0000 complete [ 7911.396809] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7911.400761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7911.416177] Lustre: Skipped 1 previous similar message [ 7911.426836] Lustre: Skipped 2 previous similar messages [ 7915.226202] Lustre: server umount lustre-MDT0000 complete [ 7916.512846] LustreError: 271708:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7916.542689] LustreError: 271708:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7919.155616] LustreError: 270450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790104387 with bad export cookie 18180440092933420913 [ 7919.164627] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7919.178473] LustreError: 270450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7919.579389] Lustre: server umount lustre-MDT0001 complete [ 7932.185918] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 7935.004644] Key type lgssc unregistered [ 7935.266662] LNet: 274055:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7935.271655] LNetError: 274055:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7935.284220] LNet: Removed LNI 192.168.203.122@tcp [ 7936.178615] Key type .llcrypt unregistered [ 7936.186753] Key type ._llcrypt unregistered [ 7953.522614] Key type ._llcrypt registered [ 7953.527636] Key type .llcrypt registered [ 7953.631472] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7954.711267] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7954.989432] alg: No test for adler32 (adler32-zlib) [ 7956.073939] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 7956.336595] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 7958.023456] Key type lgssc registered [ 7959.450647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7969.725208] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7969.762579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7985.376040] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7990.495531] LustreError: 275266:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.122@tcp: failed processing log, type 4: rc = -110 [ 8017.184824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8023.243678] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8031.318664] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8034.178539] Lustre: server umount lustre-OST0000 complete [ 8052.924967] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8055.922822] Key type lgssc unregistered [ 8056.213606] LNet: 276898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8056.227752] LNetError: 276898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8056.254906] LNet: Removed LNI 192.168.203.122@tcp [ 8057.228842] Key type .llcrypt unregistered [ 8057.237472] Key type ._llcrypt unregistered [ 8102.164853] Key type ._llcrypt registered [ 8102.166663] Key type .llcrypt registered [ 8102.306833] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8103.313949] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8103.498912] alg: No test for adler32 (adler32-zlib) [ 8104.663579] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 8104.911851] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 8106.599805] Key type lgssc registered [ 8107.863930] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8121.696263] loop0: detected capacity change from 0 to 204800000 [ 8121.914970] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8121.934530] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8122.234912] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8123.256344] loop0: detected capacity change from 0 to 204800000 [ 8123.475308] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8123.490563] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8124.051354] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8125.295392] loop0: detected capacity change from 0 to 204800000 [ 8125.468321] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8125.478309] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8125.575978] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8126.599899] loop0: detected capacity change from 0 to 204800000 [ 8126.803520] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8127.890367] loop0: detected capacity change from 0 to 204800000 [ 8127.925773] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8132.119871] loop0: detected capacity change from 0 to 204800000 [ 8132.165336] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8136.084438] loop0: detected capacity change from 0 to 204800000 [ 8136.110426] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8140.123560] loop0: detected capacity change from 0 to 204800000 [ 8140.160079] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8143.995871] loop0: detected capacity change from 0 to 204800000 [ 8144.083260] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8144.126273] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8144.269765] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8144.280586] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8144.292445] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8144.309776] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8144.323519] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8144.341513] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8144.359862] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8144.373869] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8144.387675] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8144.402354] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8144.412540] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8144.422825] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8144.430724] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8144.439498] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8149.858391] Lustre: server umount MGS complete [ 8150.779485] loop0: detected capacity change from 0 to 204800000 [ 8150.843233] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8151.373328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8151.420902] Lustre: lustre-MDT0000: Aborting client recovery [ 8152.320580] loop1: detected capacity change from 0 to 204800000 [ 8152.404629] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8152.505014] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8152.526191] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8152.541573] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8152.556788] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8152.571370] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8152.586903] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8152.598863] Lustre: 280757:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8152.635199] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8152.853535] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8152.916896] Lustre: lustre-MDT0001: Aborting client recovery [ 8153.863602] loop2: detected capacity change from 0 to 204800000 [ 8154.013858] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8154.085436] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8154.093899] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8154.103276] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8154.113463] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8154.123920] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8154.133390] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8154.144103] Lustre: 280837:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8154.264319] Lustre: Found index 0 for lustre-OST0000, updating log [ 8154.302305] Lustre: 280855:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8154.394162] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8154.414419] Lustre: lustre-OST0000: Aborting recovery [ 8155.226808] loop3: detected capacity change from 0 to 204800000 [ 8155.400546] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8155.527444] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8155.548939] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8155.563762] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8155.579511] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8155.597131] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8155.607896] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8155.616401] Lustre: 280921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8155.665990] Lustre: Found index 1 for lustre-OST0001, updating log [ 8155.706609] Lustre: 280926:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8155.787621] Lustre: lustre-OST0001: Aborting recovery [ 8160.872256] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8160.884077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8160.958438] LustreError: 280700:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8160.962141] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8160.989662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8162.935367] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8162.953900] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8163.488580] LustreError: 280667:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1002us: rc = -2 [ 8163.998779] LustreError: 280664:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 469us: rc = -2 [ 8164.015790] LustreError: 280664:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [ 8178.660046] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8178.662392] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8178.662707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8178.683442] Lustre: Skipped 2 previous similar messages [ 8182.500296] Lustre: server umount lustre-MDT0000 complete [ 8183.564318] LustreError: 280650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790104651 with bad export cookie 12285132665782680709 [ 8183.583267] LustreError: 280650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8183.584759] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8183.777931] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8183.779962] LustreError: 280665:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8183.780483] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8183.780520] Lustre: Skipped 3 previous similar messages [ 8183.784344] Lustre: Skipped 2 previous similar messages [ 8183.794611] LustreError: 280665:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8188.897632] LustreError: 280862:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8188.898976] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8188.946076] Lustre: Skipped 2 previous similar messages [ 8190.029444] Lustre: server umount lustre-MDT0001 complete [ 8205.280605] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8205.454910] Lustre: server umount lustre-OST0000 complete [ 8212.906654] Lustre: server umount lustre-OST0001 complete [ 8222.270465] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 15:18:08 (1790104688) [ 8247.316647] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8289.513386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8289.678743] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8289.721475] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8289.825104] Lustre: lustre-MDT0000: new disk, initializing [ 8289.874216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8289.884128] Lustre: Skipped 1 previous similar message [ 8289.894772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8293.938321] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8301.033733] Lustre: server umount lustre-MDT0000 complete [ 8319.465373] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 15:19:46 (1790104786) [ 8342.592985] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 8351.563742] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8383.040269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8383.212731] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8383.229623] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8383.232993] Lustre: Skipped 1 previous similar message [ 8383.309918] Lustre: lustre-MDT0000: new disk, initializing [ 8383.376052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8383.391328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8387.056424] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8396.618519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8396.737347] Lustre: 288027:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8396.788413] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8396.792454] Lustre: Skipped 1 previous similar message [ 8396.862405] Lustre: lustre-MDT0001: new disk, initializing [ 8396.960992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8397.014302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8397.027083] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8400.907726] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8409.449540] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8414.653909] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8420.678737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8420.838916] Lustre: lustre-OST0000: new disk, initializing [ 8420.841192] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8420.844915] Lustre: 289994:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8420.893886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8426.965779] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8429.131797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8429.139694] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8429.232507] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8434.853630] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8439.541218] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8453.281851] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8460.781491] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8511.463169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8511.472249] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8511.486201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8511.506489] Lustre: Skipped 1 previous similar message [ 8515.650658] Lustre: server umount lustre-OST0000 complete [ 8520.163657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8520.169125] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8520.176671] Lustre: Skipped 1 previous similar message [ 8520.181187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8525.885782] Lustre: server umount lustre-MDT0000 complete [ 8526.842163] LustreError: 288925:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8526.862280] LustreError: 288925:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8529.236911] LustreError: 288019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790104997 with bad export cookie 12285132665782683677 [ 8529.245806] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8529.249202] LustreError: 288019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8529.600920] Lustre: server umount lustre-MDT0001 complete [ 8538.179119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8542.787369] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8547.897732] Lustre: server umount MGS complete [ 8558.469038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8558.938784] LustreError: 293966:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8558.964142] LustreError: 293979:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8559.025688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8563.384923] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8564.195676] LustreError: 293967:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8564.212058] LustreError: 293967:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8571.733602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8576.392534] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8582.739635] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8588.264604] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8595.137514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8600.967449] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8608.434214] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8640.483470] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8640.490467] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8640.503643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8640.513014] Lustre: Skipped 2 previous similar messages [ 8640.525507] Lustre: Skipped 3 previous similar messages [ 8645.365281] Lustre: server umount lustre-OST0000 complete [ 8649.187711] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8649.196134] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8654.901481] Lustre: server umount lustre-MDT0000 complete [ 8655.850473] LustreError: 294695:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8655.868557] LustreError: 294695:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8658.083534] LustreError: 293946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790105126 with bad export cookie 12285132665782685616 [ 8658.086068] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8658.089246] LustreError: 293946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8658.277056] Lustre: server umount lustre-MDT0001 complete [ 8668.774513] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8671.694368] Key type lgssc unregistered [ 8671.971927] LNet: 297913:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8671.997350] LNetError: 297913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8672.020406] LNet: Removed LNI 192.168.203.122@tcp [ 8672.838381] Key type .llcrypt unregistered [ 8672.845464] Key type ._llcrypt unregistered [ 8709.814761] Key type ._llcrypt registered [ 8709.816442] Key type .llcrypt registered [ 8709.905979] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 8723.600570] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8724.715792] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8724.773091] alg: No test for adler32 (adler32-zlib) [ 8725.883600] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 8726.130131] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 8727.784156] Key type lgssc registered [ 8729.124352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8759.933825] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8759.960744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8761.260172] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8761.301488] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8761.396562] Lustre: lustre-MDT0000: new disk, initializing [ 8761.452736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8761.473782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8765.817303] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8775.808902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8775.921434] Lustre: 301877:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8776.008512] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8776.012234] Lustre: Skipped 1 previous similar message [ 8776.089220] Lustre: lustre-MDT0001: new disk, initializing [ 8776.197395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8776.230907] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8776.242885] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8781.009814] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8791.274660] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8797.061834] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8803.649317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8803.973614] Lustre: lustre-OST0000: new disk, initializing [ 8803.976763] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8803.982703] Lustre: 303842:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8804.108563] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8809.569538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8809.579911] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8809.721208] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8810.619371] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8820.625601] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8826.908546] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8840.787524] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8847.792387] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8896.998759] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8897.000183] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8897.005136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8897.009929] LustreError: Skipped 1 previous similar message [ 8897.039772] Lustre: Skipped 1 previous similar message [ 8902.118150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8902.122064] Lustre: Skipped 1 previous similar message [ 8907.245137] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8907.252788] Lustre: Skipped 1 previous similar message [ 8908.255157] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8908.396136] Lustre: server umount lustre-OST0000 complete [ 8912.354256] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8912.367331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8912.381092] Lustre: Skipped 1 previous similar message [ 8917.537112] Lustre: server umount lustre-MDT0000 complete [ 8921.163502] LustreError: 304699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790105389 with bad export cookie 13902860730446692458 [ 8921.166119] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8921.170129] LustreError: 304699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8921.429947] Lustre: server umount lustre-MDT0001 complete [ 8928.790858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8932.256458] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8935.625945] Lustre: server umount MGS complete [ 8944.599524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8944.953418] LustreError: 307855:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8944.970880] LustreError: 307855:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8944.982253] LustreError: 307868:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8945.035430] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8948.635515] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8950.240800] LustreError: 307856:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8950.262812] LustreError: 307856:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8955.270553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8955.360301] LustreError: 307855:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8955.601337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8959.068935] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8965.656891] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8971.054805] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8978.033664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8978.479273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8985.172302] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8991.930857] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9018.847864] Lustre: server umount lustre-OST0000 complete [ 9022.433631] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9022.440453] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9022.450649] Lustre: Skipped 1 previous similar message [ 9022.457732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9025.508211] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9030.626053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9030.643323] Lustre: Skipped 3 previous similar messages [ 9031.734671] Lustre: server umount lustre-MDT0000 complete [ 9035.748928] LustreError: 308950:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9035.774838] LustreError: 308950:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9036.092306] LustreError: 307836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790105504 with bad export cookie 13902860730446694299 [ 9036.098034] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9036.115830] LustreError: 307836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9046.797542] Lustre: server umount lustre-MDT0001 complete [ 9056.556099] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 9058.988602] Key type lgssc unregistered [ 9059.241360] LNet: 311799:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9059.250714] LNetError: 311799:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9059.267661] LNet: Removed LNI 192.168.203.122@tcp [ 9060.139159] Key type .llcrypt unregistered [ 9060.143842] Key type ._llcrypt unregistered [ 9082.764320] Key type ._llcrypt registered [ 9082.765820] Key type .llcrypt registered [ 9082.918738] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 9098.127496] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9099.295198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9099.491048] alg: No test for adler32 (adler32-zlib) [ 9100.755820] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [ 9101.050651] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 9102.719223] Key type lgssc registered [ 9103.787662] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9129.109874] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9142.235443] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9142.250510] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9143.480709] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9143.516115] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9143.613933] Lustre: lustre-MDT0000: new disk, initializing [ 9143.687544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9143.718688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9147.396189] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9156.721919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9156.994750] Lustre: lustre-OST0000: new disk, initializing [ 9157.000442] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9157.014118] Lustre: Skipped 1 previous similar message [ 9157.022877] Lustre: 315864:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9157.093278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9160.769768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9160.783409] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9160.811578] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9162.555489] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10025.718313] Lustre: server umount lustre-MDT0000 complete [10028.611364] LustreError: 315050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790106496 with bad export cookie 5644681537215513548 [10028.639732] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10038.365456] Lustre: server umount lustre-OST0000 complete [10058.414453] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [10067.210930] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10094.623928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10094.764637] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10094.786601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10094.838730] Lustre: lustre-MDT0000: new disk, initializing [10094.915655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10094.939884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10098.248279] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10105.912627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10105.987984] Lustre: 320203:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10106.030745] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10106.036092] Lustre: Skipped 1 previous similar message [10106.079539] Lustre: lustre-MDT0001: new disk, initializing [10106.170764] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10106.199819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10106.205896] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10109.401384] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10116.706521] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10121.097776] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10126.073826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10126.294424] Lustre: lustre-OST0000: new disk, initializing [10126.298575] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10126.304328] Lustre: 322168:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10126.382728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10131.015082] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10134.575689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10134.593176] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10134.693054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10138.744307] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10142.714786] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10142.887646] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10146.575566] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10146.749340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10150.369029] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10150.381024] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10150.384290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10150.386042] LustreError: Skipped 1 previous similar message [10150.400658] Lustre: Skipped 1 previous similar message [10154.403378] Lustre: server umount lustre-OST0000 complete [10157.536308] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10157.542720] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10157.552842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10157.556976] Lustre: Skipped 1 previous similar message [10160.608990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10160.610777] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10163.553177] Lustre: server umount lustre-MDT0000 complete [10166.449706] LustreError: 320193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790106634 with bad export cookie 5644681537218454367 [10166.455237] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10166.462315] LustreError: 320193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10166.702976] Lustre: server umount lustre-MDT0001 complete [10169.176508] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10170.308566] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 15:50:37 (1790106637) [10174.571765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10174.866120] LustreError: 324478:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10174.875755] LustreError: 324478:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10174.902607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10177.711070] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10180.066072] LustreError: 324479:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10183.814937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10184.046362] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10187.419063] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10192.366226] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10196.639126] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10202.032686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10202.430597] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10207.357969] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10213.006409] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10218.433946] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10218.638492] Lustre: lustre-OST0001: new disk, initializing [10218.645695] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10218.652251] Lustre: 327311:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10218.745056] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10223.336049] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10226.248570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10226.255924] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10226.293191] LustreError: 327796:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10227.374952] LustreError: 327796:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10227.383599] LustreError: 327796:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10228.386072] LustreError: 327806:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10230.434085] LustreError: 327806:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10230.447415] LustreError: 327806:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10231.606875] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10231.806964] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10234.598903] LustreError: 327796:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10234.609693] LustreError: 327796:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10236.517830] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c1b883f6000.ost_server_uuid 50 [10237.771926] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c1b883f6000.ost_server_uuid in FULL state after 0 sec [10242.218285] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c1b883f6000.ost_server_uuid 50 [10242.794918] LustreError: 327806:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10242.800661] LustreError: 327806:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10243.571902] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c1b883f6000.ost_server_uuid in FULL state after 0 sec [10244.861748] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10258.374231] Lustre: lustre-OST0001: Not available for connect from 192.168.203.22@tcp (stopping) [10258.378852] Lustre: Skipped 1 previous similar message [10260.453715] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10260.455383] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10260.467454] LustreError: Skipped 1 previous similar message [10262.503087] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10262.515780] Lustre: Skipped 3 previous similar messages [10272.223629] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10272.323626] Lustre: server umount lustre-OST0001 complete [10272.742168] LustreError: 327887:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10272.759290] LustreError: 327887:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10277.861792] LustreError: 327391:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10277.865532] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10277.873465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10277.873474] Lustre: Skipped 4 previous similar messages [10277.890643] LustreError: 327391:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10277.932438] Lustre: Skipped 2 previous similar messages [10282.968152] Lustre: server umount lustre-OST0000 complete [10286.561850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10286.572520] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10298.337341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10298.340537] Lustre: Skipped 6 previous similar messages [10300.383151] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10302.610538] Lustre: server umount lustre-MDT0000 complete [10303.463486] LustreError: 324473:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10303.474622] LustreError: 324473:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10305.473179] LustreError: 324458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790106773 with bad export cookie 5644681537218455242 [10305.475547] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10305.481360] LustreError: 324458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10305.797108] Lustre: server umount lustre-MDT0001 complete [10313.959312] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [10316.096294] Key type lgssc unregistered [10316.371569] LNet: 329998:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10316.386278] LNetError: 329998:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10317.415747] LNet: Removed LNI 192.168.203.122@tcp [10318.152396] Key type .llcrypt unregistered [10318.156297] Key type ._llcrypt unregistered [10338.615953] Key type ._llcrypt registered [10338.618794] Key type .llcrypt registered [10338.708814] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10339.400200] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10339.590993] alg: No test for adler32 (adler32-zlib) [10340.579556] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [10340.770711] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [10342.423223] Key type lgssc registered [10343.408620] Lustre: Echo OBD driver; http://www.lustre.org/ [10348.566970] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10348.590099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10349.913507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10352.746852] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10357.659787] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10360.297290] LustreError: 331416:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10360.311325] LustreError: 331416:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10362.086582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10362.315595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10365.313481] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10372.139263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10372.378958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10373.416828] LustreError: 333095:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10375.470392] LustreError: 333184:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10375.481698] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10375.484265] LustreError: 333184:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10377.132625] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10380.775188] LustreError: 333184:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10380.791248] LustreError: 333184:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10382.699144] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10385.896414] LustreError: 333097:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10385.910320] LustreError: 333097:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10387.430188] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10387.601168] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10391.570981] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10393.081193] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10396.453033] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10400.719450] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10400.960684] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10495.461570] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10495.485211] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10495.485767] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10495.493587] Lustre: Skipped 1 previous similar message [10500.582279] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10501.477082] Lustre: server umount lustre-OST0001 complete [10505.703652] LustreError: 333097:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.718198] LustreError: 333097:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10506.723486] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10506.732127] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10506.746446] Lustre: Skipped 1 previous similar message [10506.754402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10506.760023] Lustre: Skipped 1 previous similar message [10508.770557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10508.791970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10508.803248] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10515.938698] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10515.947994] Lustre: Skipped 3 previous similar messages [10520.031128] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10520.163787] Lustre: server umount lustre-OST0000 complete [10526.179230] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10526.183927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10526.184156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10526.193492] Lustre: Skipped 1 previous similar message [10531.260555] Lustre: server umount lustre-MDT0000 complete [10531.302287] LustreError: 331417:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10531.316193] LustreError: 331417:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [10534.125550] LustreError: 331398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790107002 with bad export cookie 6739363227061446853 [10534.140889] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10534.147755] LustreError: 331398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10534.384418] Lustre: server umount lustre-MDT0001 complete [10543.067805] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [10545.301866] Key type lgssc unregistered [10545.488742] LNet: 336356:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10545.497920] LNetError: 336356:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10545.514563] LNet: Removed LNI 192.168.203.122@tcp [10546.186384] Key type .llcrypt unregistered [10546.188277] Key type ._llcrypt unregistered [10583.283216] Key type ._llcrypt registered [10583.285200] Key type .llcrypt registered [10583.368700] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10584.095288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10584.121106] alg: No test for adler32 (adler32-zlib) [10585.119529] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [10585.335072] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [10586.999182] Key type lgssc registered [10587.944445] Lustre: Echo OBD driver; http://www.lustre.org/ [10594.939981] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10594.965293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10596.369830] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10599.579287] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10606.310936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10606.655160] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10611.369321] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10614.327147] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10620.386290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10620.870129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10621.882327] LustreError: 340142:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10621.914319] LustreError: 340142:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10621.924743] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10626.920605] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10627.049706] LustreError: 340142:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10631.139319] LustreError: 340141:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10631.167307] LustreError: 340141:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10635.091400] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10635.385629] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10640.886251] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10641.001208] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10649.616775] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10660.471310] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10671.587720] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10671.590652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10671.605560] Lustre: Skipped 2 previous similar messages [10671.619265] Lustre: Skipped 3 previous similar messages [10676.717746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10676.724873] Lustre: Skipped 3 previous similar messages [10676.813218] Lustre: server umount lustre-MDT0000 complete [10679.878365] LustreError: 341659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790107148 with bad export cookie 9042493867491222990 [10679.889409] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10679.895576] LustreError: 341659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10680.111432] Lustre: server umount lustre-MDT0001 complete [10693.356387] Lustre: server umount lustre-OST0000 complete [10706.529334] Lustre: server umount lustre-OST0001 complete [10718.495465] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10726.100338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10726.351256] LustreError: 343917:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10726.410204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10729.468734] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10731.498270] LustreError: 343918:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10735.635417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10735.891748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10739.274624] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10741.480203] Lustre: 345059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10746.187339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10750.452773] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10752.559798] LustreError: 345413:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10752.578210] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10757.060729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10757.285121] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10757.293581] Lustre: Skipped 1 previous similar message [10761.401789] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10761.804631] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10767.692459] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10770.268954] Lustre: 346929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10777.065076] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10777.065812] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10777.067525] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10777.072779] Lustre: Skipped 3 previous similar messages [10778.960264] Lustre: server umount lustre-MDT0000 complete [10781.267645] LustreError: 343897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790107249 with bad export cookie 9042493867491224691 [10781.268320] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10781.276329] LustreError: 343897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10781.504056] Lustre: server umount lustre-MDT0001 complete [10794.634892] Lustre: server umount lustre-OST0000 complete [10806.962661] Lustre: server umount lustre-OST0001 complete [10809.826137] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10810.957514] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10812.080442] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 16:01:19 (1790107279) [10829.423022] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10837.200517] loop0: detected capacity change from 0 to 1099511627776 [10837.672845] SGI XFS with ACLs, security attributes, quota, no debug enabled [10837.679063] *** VALIDATE xfs *** [10837.707060] XFS (loop0): Mounting V5 Filesystem [10837.740496] XFS (loop0): Ending clean mount [10838.845578] loop1: detected capacity change from 0 to 18691697672192 [10916.352341] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [10924.592811] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10964.472940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10964.813752] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10964.856378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10964.958850] Lustre: lustre-MDT0000: new disk, initializing [10965.100583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10965.120742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10972.091196] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10988.030278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10988.177622] Lustre: 352066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10988.298496] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10988.304916] Lustre: Skipped 1 previous similar message [10988.451501] Lustre: lustre-MDT0001: new disk, initializing [10989.073916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10989.115971] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10994.419781] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11006.664175] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11012.721551] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11019.478044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11019.836943] Lustre: lustre-OST0000: new disk, initializing [11019.844678] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11019.857691] Lustre: 354039:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11021.358515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11021.373051] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11021.483344] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11025.875789] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11034.835602] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11039.094774] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11039.317249] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11043.423515] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11043.654932] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11047.395474] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11047.396113] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11047.417148] Lustre: Skipped 1 previous similar message [11047.418877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11047.445643] Lustre: Skipped 4 previous similar messages [11051.879354] Lustre: server umount lustre-OST0000 complete [11057.634392] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11057.636627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11057.658459] Lustre: Skipped 2 previous similar messages [11057.673753] Lustre: Skipped 1 previous similar message [11061.842788] Lustre: server umount lustre-MDT0000 complete [11062.754218] LustreError: 352079:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11062.773626] LustreError: 352079:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11065.429595] LustreError: 352058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790107533 with bad export cookie 9042493867491226238 [11065.432535] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11065.438718] LustreError: 352058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11065.811418] Lustre: server umount lustre-MDT0001 complete [11073.312335] XFS (loop0): Unmounting Filesystem [11078.540605] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 16:05:44 (1790107544) [11086.866590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11087.730917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11087.756558] Lustre: Skipped 2 previous similar messages [11092.839353] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11097.066803] LustreError: 356599:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11097.082123] LustreError: 356599:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11102.488306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11107.627191] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11115.539098] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11121.078727] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11128.212816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11134.913230] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11143.011661] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11158.497933] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11158.502451] LustreError: Skipped 1 previous similar message [11158.509518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11158.535713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11160.344565] Lustre: server umount lustre-OST0000 complete [11164.129106] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11164.137063] LustreError: Skipped 1 previous similar message [11168.743769] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11168.759348] Lustre: Skipped 2 previous similar messages [11170.412551] Lustre: server umount lustre-MDT0000 complete [11173.862335] LustreError: 356594:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11173.876131] LustreError: 356594:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11173.975980] LustreError: 356580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790107642 with bad export cookie 9042493867491227113 [11173.980785] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11173.988841] LustreError: 356580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11174.270795] Lustre: server umount lustre-MDT0001 complete [11185.181398] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [11187.741690] Key type lgssc unregistered [11187.993448] LNet: 360225:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11188.008079] LNetError: 360225:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11189.035689] LNet: Removed LNI 192.168.203.122@tcp [11189.933349] Key type .llcrypt unregistered [11189.939164] Key type ._llcrypt unregistered [11217.843745] Key type ._llcrypt registered [11217.845778] Key type .llcrypt registered [11217.974531] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11219.435096] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11219.500046] alg: No test for adler32 (adler32-zlib) [11220.571884] Lustre: Lustre: Build Version: 2.17.58_63_gba3a43f [11220.826457] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [11222.545680] Key type lgssc registered [11223.710878] Lustre: Echo OBD driver; http://www.lustre.org/ [11230.634449] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11230.670086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11232.325866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11236.903459] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11242.467118] LustreError: 361806:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11242.488937] LustreError: 361806:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11245.617102] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11246.109469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11250.668356] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11258.268122] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11265.001720] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11272.618730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11273.046544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11279.290842] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11286.973935] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11296.739605] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11296.749412] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11296.755738] Lustre: Skipped 1 previous similar message [11298.779867] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.22@tcp (stopping) [11301.858773] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11301.867152] Lustre: Skipped 1 previous similar message [11302.974044] Lustre: server umount lustre-MDT0001 complete [11303.867477] LustreError: 362810:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11306.984289] LustreError: 361807:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11307.002168] LustreError: 361807:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11312.098868] LustreError: 361802:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11312.125471] LustreError: 361802:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11312.807873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11312.937158] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11312.941775] Lustre: Found index 1 for lustre-MDT0001, updating log [11313.000604] Lustre: 361796:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11313.329270] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11317.933921] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11318.762869] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11322.886233] LustreError: 365406:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11322.896262] LustreError: 365406:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11322.910607] LustreError: 361799:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11322.981837] LustreError: 365409:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -17 [11322.996748] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11322.996748] [11623.912286] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11623.918931] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11623.932557] Lustre: Skipped 2 previous similar messages [11623.943550] Lustre: Skipped 1 previous similar message [11629.028133] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11629.040050] Lustre: Skipped 2 previous similar messages [11638.239301] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11638.403953] Lustre: server umount lustre-MDT0001 complete [11639.276191] LustreError: 366025:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11639.298530] LustreError: 366025:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11648.776430] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11648.919438] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11648.923516] Lustre: Found index 1 for lustre-MDT0001, updating log [11648.963730] Lustre: 361797:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11649.325265] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11653.526333] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11654.638888] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11655.672622] LustreError: 374252:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11655.684781] LustreError: 374252:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11655.697723] LustreError: 361799:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11655.705499] LustreError: 361799:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11655.773098] LustreError: 374255:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11655.784543] LustreError: 374255:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11655.792493] LustreError: 374255:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -17 [11655.799112] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.122@tcp [11655.799112] [11959.254216] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.22@tcp (stopping) [11959.264978] Lustre: Skipped 4 previous similar messages [11961.825673] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11961.844965] Lustre: Skipped 1 previous similar message [11961.856538] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11973.087984] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11973.279934] Lustre: server umount lustre-MDT0001 complete [11974.579974] LustreError: 361801:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11974.604779] LustreError: 361801:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [11982.987548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11983.156829] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11983.165716] Lustre: Found index 1 for lustre-MDT0001, updating log [11983.277405] Lustre: 361796:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11983.645469] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11988.451303] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11988.992108] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11993.081105] LustreError: 382858:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11993.098198] LustreError: 382858:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11993.119683] LustreError: 361799:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11993.128029] LustreError: 361799:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11993.209871] LustreError: 382861:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11993.235797] LustreError: 382861:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -114 [11993.250226] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11993.250226] [12311.522116] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12311.524214] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12311.524875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12311.524881] Lustre: Skipped 8 previous similar messages [12311.540597] Lustre: Skipped 1 previous similar message [12316.551292] Lustre: server umount lustre-MDT0000 complete [12316.647807] LustreError: 368844:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12316.665023] LustreError: 368844:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [12319.651055] LustreError: 361788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790108787 with bad export cookie 13628181483911868047 [12319.651728] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12319.665584] LustreError: 361788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12321.764875] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12321.783399] Lustre: Skipped 1 previous similar message [12326.061778] Lustre: server umount lustre-MDT0001 complete [12329.306558] Lustre: server umount lustre-OST0000 complete [12336.719110] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [12344.424760] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [12373.595548] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 16:27:20 (1790108840) [12378.668336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12378.879623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12378.906383] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12378.980792] Lustre: lustre-MDT0000: new disk, initializing [12379.039777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12379.059057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12382.395470] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12389.998746] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12390.096142] Lustre: 393786:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12390.136671] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12390.140456] Lustre: Skipped 1 previous similar message [12390.200786] Lustre: lustre-MDT0001: new disk, initializing [12390.280973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12390.310326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12390.322287] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12393.504421] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12400.884971] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12405.011607] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12410.133973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12410.406862] Lustre: lustre-OST0000: new disk, initializing [12410.410070] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12410.416359] Lustre: 395756:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12410.484860] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12415.020719] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12417.043961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12417.054805] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12417.103190] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12422.459630] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12430.823680] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12430.830496] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12430.837707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12430.844100] Lustre: Skipped 4 previous similar messages [12434.246474] Lustre: server umount lustre-MDT0000 complete [12436.621874] LustreError: 393780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790108904 with bad export cookie 13628181483911870000 [12436.626185] LustreError: 394594:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12436.628778] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12436.632270] LustreError: 393780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12436.878326] Lustre: server umount lustre-MDT0001 complete [12449.775974] Lustre: server umount lustre-OST0000 complete [12469.763034] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [12476.356383] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [12497.446326] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12189 sec ========= 16:29:24 (1790108964) [12498.592073] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 16:29:25 (1790108965) === [12500.906382] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 16:29:28 (1790108968) === [12526.268496] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [12528.047087] Key type lgssc unregistered [12528.243908] LNet: 402110:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12528.249677] LNetError: 402110:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12528.266314] LNet: Removed LNI 192.168.203.122@tcp [12528.880888] Key type .llcrypt unregistered [12528.883573] Key type ._llcrypt unregistered