[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 462088128 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001016] APIC: Switch to symmetric I/O mode setup [ 0.002364] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004018] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007042] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008022] pid_max: default: 32768 minimum: 301 [ 0.009185] LSM: Security Framework initializing [ 0.010090] Yama: becoming mindful. [ 0.012059] SELinux: Initializing. [ 0.013090] *** VALIDATE selinux *** [ 0.022450] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028004] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029183] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030138] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032137] *** VALIDATE tmpfs *** [ 0.034190] *** VALIDATE proc *** [ 0.035297] *** VALIDATE cgroup *** [ 0.036018] *** VALIDATE cgroup2 *** [ 0.038182] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.040104] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.041014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042041] Spectre V2 : User space: Vulnerable [ 0.043014] Speculative Store Bypass: Vulnerable [ 0.046591] debug: unmapping init [mem 0xffffffff9c459000-0xffffffff9c460fff] [ 0.049179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050779] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051024] ... version: 2 [ 0.052017] ... bit width: 48 [ 0.053017] ... generic registers: 4 [ 0.054021] ... value mask: 0000ffffffffffff [ 0.055029] ... max period: 00007fffffffffff [ 0.056018] ... fixed-purpose events: 3 [ 0.057013] ... event mask: 000000070000000f [ 0.058315] rcu: Hierarchical SRCU implementation. [ 0.060521] smp: Bringing up secondary CPUs ... [ 0.061657] x86: Booting SMP configuration: [ 0.062043] .... node #0, CPUs: #1 #2 #3 [ 0.070578] smp: Brought up 1 node, 4 CPUs [ 0.072015] smpboot: Max logical packages: 1 [ 0.073016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.191027] node 0 deferred pages initialised in 116ms [ 0.196041] devtmpfs: initialized [ 0.198304] x86/mm: Memory block size: 128MB [ 0.201000] gcov: version magic: 0x41383552 [ 0.204335] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.208080] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.211323] pinctrl core: initialized pinctrl subsystem [ 0.213196] [ 0.213847] ************************************************************* [ 0.215015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.218015] ** ** [ 0.221018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.224015] ** ** [ 0.227018] ** This means that this kernel is built to expose internal ** [ 0.229015] ** IOMMU data structures, which may compromise security on ** [ 0.231011] ** your system. ** [ 0.233014] ** ** [ 0.235015] ** If you see this message and you are not debugging the ** [ 0.237013] ** kernel, report this immediately to your vendor! ** [ 0.238009] ** ** [ 0.240016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.243015] ************************************************************* [ 0.245601] NET: Registered protocol family 16 [ 0.247441] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.250061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.253071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.257182] cpuidle: using governor menu [ 0.259000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.261789] PCI: Using configuration type 1 for base access [ 0.264115] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.275134] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.278123] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.283030] cryptd: max_cpu_qlen set to 1000 [ 0.286251] ACPI: Added _OSI(Module Device) [ 0.288040] ACPI: Added _OSI(Processor Device) [ 0.289019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.291015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.296270] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.304424] ACPI: Interpreter enabled [ 0.306073] ACPI: PM: (supports S0 S3 S4 S5) [ 0.307022] ACPI: Using IOAPIC for interrupt routing [ 0.309393] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.315419] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.326832] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.328035] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.330017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.333076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.337294] acpiphp: Slot [2] registered [ 0.338096] acpiphp: Slot [5] registered [ 0.339148] acpiphp: Slot [6] registered [ 0.340082] acpiphp: Slot [7] registered [ 0.341106] acpiphp: Slot [8] registered [ 0.342098] acpiphp: Slot [9] registered [ 0.344096] acpiphp: Slot [10] registered [ 0.345083] acpiphp: Slot [3] registered [ 0.346069] acpiphp: Slot [4] registered [ 0.347071] acpiphp: Slot [11] registered [ 0.348093] acpiphp: Slot [12] registered [ 0.349072] acpiphp: Slot [13] registered [ 0.350150] acpiphp: Slot [14] registered [ 0.351075] acpiphp: Slot [15] registered [ 0.352072] acpiphp: Slot [16] registered [ 0.353066] acpiphp: Slot [17] registered [ 0.354068] acpiphp: Slot [18] registered [ 0.355080] acpiphp: Slot [19] registered [ 0.356071] acpiphp: Slot [20] registered [ 0.357090] acpiphp: Slot [21] registered [ 0.358134] acpiphp: Slot [22] registered [ 0.360079] acpiphp: Slot [23] registered [ 0.361078] acpiphp: Slot [24] registered [ 0.362072] acpiphp: Slot [25] registered [ 0.362953] acpiphp: Slot [26] registered [ 0.364089] acpiphp: Slot [27] registered [ 0.365087] acpiphp: Slot [28] registered [ 0.367296] acpiphp: Slot [29] registered [ 0.369126] acpiphp: Slot [30] registered [ 0.371238] acpiphp: Slot [31] registered [ 0.373064] PCI host bridge to bus 0000:00 [ 0.374018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.376025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.379025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.382029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.385026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.387026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.390181] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.392291] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.396509] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.407806] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.411976] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.415027] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.417019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.419018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.422603] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.426736] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.429061] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.431610] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.436017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.449900] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.453908] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.459531] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.465013] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.470015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.487016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.496916] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.509017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.518019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.538018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.548870] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.554014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.560015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.573013] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.582015] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.589015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.593017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.607017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.617576] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.623016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.637016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.655019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.665335] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.678017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.685017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.706015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.717685] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.720369] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.723252] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.725257] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.727211] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.733035] iommu: Default domain type: Passthrough [ 0.734385] SCSI subsystem initialized [ 0.736149] ACPI: bus type USB registered [ 0.738076] usbcore: registered new interface driver usbfs [ 0.739077] usbcore: registered new interface driver hub [ 0.740080] usbcore: registered new device driver usb [ 0.743213] pps_core: LinuxPPS API ver. 1 registered [ 0.745013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.748063] PTP clock support registered [ 0.751040] EDAC MC: Ver: 3.0.0 [ 0.753129] PCI: Using ACPI for IRQ routing [ 0.755558] NetLabel: Initializing [ 0.757015] NetLabel: domain hash size = 128 [ 0.759012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.761078] NetLabel: unlabeled traffic allowed by default [ 0.766163] vgaarb: loaded [ 0.768221] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.769014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.773850] clocksource: Switched to clocksource kvm-clock [ 0.882695] VFS: Disk quotas dquot_6.6.0 [ 0.884644] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.887349] *** VALIDATE ramfs *** [ 0.889078] *** VALIDATE hugetlbfs *** [ 0.891766] pnp: PnP ACPI init [ 0.894196] pnp: PnP ACPI: found 6 devices [ 0.910308] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.913823] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.916028] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.918457] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.920948] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.922995] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.928238] NET: Registered protocol family 2 [ 0.930753] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.935224] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.938733] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.943374] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.947048] TCP: Hash tables configured (established 65536 bind 65536) [ 0.950295] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.953854] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.956847] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.959914] NET: Registered protocol family 1 [ 0.962618] RPC: Registered named UNIX socket transport module. [ 0.965181] RPC: Registered udp transport module. [ 0.967070] RPC: Registered tcp transport module. [ 0.968728] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.971143] NET: Registered protocol family 44 [ 0.972763] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.974964] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.977118] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.979310] PCI: CLS 0 bytes, default 64 [ 0.980839] Unpacking initramfs... [ 2.379510] debug: unmapping init [mem 0xffff8e70fcc54000-0xffff8e70fffbffff] [ 2.383575] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.386013] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.389031] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.906773] Initialise system trusted keyrings [ 2.908779] Key type blacklist registered [ 2.910655] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.923338] zbud: loaded [ 2.926966] *** VALIDATE nfs *** [ 2.929665] *** VALIDATE nfs4 *** [ 2.931524] pstore: using deflate compression [ 2.934983] Platform Keyring initialized [ 3.036135] NET: Registered protocol family 38 [ 3.038246] Key type asymmetric registered [ 3.040038] Asymmetric key parser 'x509' registered [ 3.042237] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.045815] io scheduler mq-deadline registered [ 3.047968] io scheduler kyber registered [ 3.049719] io scheduler bfq registered [ 3.052593] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.056446] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.059937] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.063191] ACPI: Power Button [PWRF] [ 3.068854] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.077121] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.096740] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.106742] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.122470] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.151372] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.179481] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.184297] Non-volatile memory driver v1.3 [ 3.185965] Linux agpgart interface v0.103 [ 3.215559] virtio_blk virtio1: [vda] 146608 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.218546] vda: detected capacity change from 0 to 75063296 [ 3.232291] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.235170] vdb: detected capacity change from 0 to 1073741824 [ 3.248748] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.251639] vdc: detected capacity change from 0 to 2621440000 [ 3.266393] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.270261] vdd: detected capacity change from 0 to 2621440000 [ 3.292965] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.296078] vde: detected capacity change from 0 to 4294967296 [ 3.309732] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.313093] vdf: detected capacity change from 0 to 4294967296 [ 3.319230] libphy: Fixed MDIO Bus: probed [ 3.323951] usbcore: registered new interface driver usbserial_generic [ 3.326947] usbserial: USB Serial support registered for generic [ 3.329300] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.333891] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.335472] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.338179] mousedev: PS/2 mouse device common for all mice [ 3.341451] rtc_cmos 00:05: RTC can wake from S4 [ 3.341883] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.346567] rtc_cmos 00:05: registered as rtc0 [ 3.352245] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.355471] intel_pstate: CPU model not supported [ 3.360077] hid: raw HID events driver (C) Jiri Kosina [ 3.362768] usbcore: registered new interface driver usbhid [ 3.366661] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.368283] usbhid: USB HID core driver [ 3.368451] drop_monitor: Initializing network drop monitor service [ 3.368573] Initializing XFRM netlink socket [ 3.368960] NET: Registered protocol family 10 [ 3.375594] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.379407] Segment Routing with IPv6 [ 3.385600] NET: Registered protocol family 17 [ 3.387590] mpls_gso: MPLS GSO support [ 3.394203] RAS: Correctable Errors collector initialized. [ 3.396335] AVX version of gcm_enc/dec engaged. [ 3.397877] AES CTR mode by8 optimization enabled [ 3.470832] sched_clock: Marking stable (3470810454, 0)->(4459106617, -988296163) [ 3.474683] registered taskstats version 1 [ 3.476581] Loading compiled-in X.509 certificates [ 3.478328] zswap: loaded using pool lzo/zbud [ 3.499728] Key type big_key registered [ 3.512985] Key type encrypted registered [ 3.514742] ima: No TPM chip found, activating TPM-bypass! [ 3.516498] ima: Allocated hash algorithm: sha1 [ 3.518070] ima: No architecture policies found [ 3.519945] evm: Initialising EVM extended attributes: [ 3.521810] evm: security.selinux [ 3.523080] evm: security.ima [ 3.524193] evm: security.capability [ 3.525674] evm: HMAC attrs: 0x1 [ 3.528481] rtc_cmos 00:05: setting system clock to 2026-08-26 23:25:06 UTC (1787786706) [ 3.535729] debug: unmapping init [mem 0xffffffff9d403000-0xffffffff9d5fffff] [ 3.538785] debug: unmapping init [mem 0xffffffff9c182000-0xffffffff9c458fff] [ 3.547149] Write protecting the kernel read-only data: 28672k [ 3.551041] debug: unmapping init [mem 0xffffffff9a803000-0xffffffff9a9fffff] [ 3.554224] debug: unmapping init [mem 0xffffffff9b114000-0xffffffff9b1fffff] [ 3.588925] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.599866] systemd[1]: Detected virtualization kvm. [ 3.601892] systemd[1]: Detected architecture x86-64. [ 3.604046] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.634534] systemd[1]: No hostname configured. [ 3.636469] systemd[1]: Set hostname to . [ 3.638571] random: systemd: uninitialized urandom read (16 bytes read) [ 3.641094] systemd[1]: Initializing machine ID from random generator. [ 3.689738] random: ln: uninitialized urandom read (6 bytes read) [ 3.782213] random: systemd: uninitialized urandom read (16 bytes read) [ 3.785226] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.789659] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.796163] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.455489] device-mapper: uevent: version 1.0.3 [ 4.457673] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.242087] virtio_net virtio0 ens2: renamed from eth0 [ 5.270816] random: fast init done [ 5.284835] scsi host0: ata_piix [ 5.296636] scsi host1: ata_piix [ 5.299497] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.303513] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.055412] random: crng init done [ 10.074292] random: 7 urandom warning(s) missed due to ratelimiting [ 10.295893] dracut-initqueue[587]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ 12.048308] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Reached target Remote File Systems. [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.005197] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.865049] SELinux: Disabled at runtime. [ 15.948639] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.959767] systemd[1]: Detected virtualization kvm. [ 15.963878] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.669337] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.678176] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.691581] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.696735] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.701820] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.712276] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.744866] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Startin[ 17.871857] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS g Apply Kernel Variables... [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 19.450334] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.550656] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 20.651514] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.255308] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.313618] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 26.406458] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.231342] NFS: Registering the id_resolver key type [ 27.235269] Key type id_resolver registered [ 27.237532] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg644-server login: [ 72.445175] hrtimer: interrupt took 4736646 ns [ 86.477639] libcfs: loading out-of-tree module taints kernel. [ 86.614143] Key type ._llcrypt registered [ 86.615791] Key type .llcrypt registered [ 86.755845] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 104.246310] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 106.016356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 106.034439] alg: No test for adler32 (adler32-zlib) [ 107.606702] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 108.512986] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 110.295291] Key type lgssc registered [ 112.105740] Lustre: Echo OBD driver; http://www.lustre.org/ [ 126.952051] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 167.393981] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 180.166530] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 180.221273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 181.456918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 181.506595] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 181.609272] Lustre: lustre-MDT0000: new disk, initializing [ 181.713259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 181.728882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 185.991268] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 198.025980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 198.136110] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 198.168944] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 198.172745] Lustre: Skipped 1 previous similar message [ 198.270725] Lustre: lustre-MDT0001: new disk, initializing [ 198.356896] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 198.397594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 198.416579] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.633830] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.352237] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 216.396989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 216.740835] Lustre: lustre-OST0000: new disk, initializing [ 216.747782] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 216.753080] Lustre: 8447:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 216.813540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 219.997698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 220.007947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 220.140465] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 222.630709] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.039804] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 236.218885] Lustre: lustre-OST0001: new disk, initializing [ 236.232649] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 236.247736] Lustre: 9520:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 236.334249] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 241.576906] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.807455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 245.819826] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 245.872345] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 252.868230] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 263.259397] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 269.304633] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing check_logdir /tmp/testlogs/ [ 274.323520] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing yml_node [ 278.256868] Lustre: DEBUG MARKER: Client: 2.17.57.52 [ 280.871684] Lustre: DEBUG MARKER: MDS: 2.17.57.52 [ 283.585868] Lustre: DEBUG MARKER: OSS: 2.17.57.52 [ 285.303393] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 26 19:29:46 EDT 2026 [ 301.263630] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 302.638981] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 321.507913] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 321.520230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 321.549507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 322.527779] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 322.531417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 322.555869] Lustre: Skipped 2 previous similar messages [ 322.564362] Lustre: Skipped 2 previous similar messages [ 324.535383] Lustre: server umount lustre-MDT0000 complete [ 328.505116] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787031 with bad export cookie 7121502201919849624 [ 328.512203] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 328.514200] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 332.772213] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 332.773641] LustreError: 6516:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 332.788601] Lustre: Skipped 1 previous similar message [ 332.789721] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 332.789726] Lustre: Skipped 1 previous similar message [ 332.796755] LustreError: 6516:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 333.797796] LustreError: 6515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 333.818858] LustreError: 6515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 334.907510] Lustre: server umount lustre-MDT0001 complete [ 338.747696] Lustre: server umount lustre-OST0000 complete [ 342.344379] Lustre: server umount lustre-OST0001 complete [ 348.558759] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 357.451417] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 393.304749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 393.625620] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 393.671538] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 393.804901] Lustre: lustre-MDT0000: new disk, initializing [ 393.930188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 393.955889] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 399.904578] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 411.243112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.408904] Lustre: 15070:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 411.539856] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 411.542975] Lustre: Skipped 1 previous similar message [ 411.639219] Lustre: lustre-MDT0001: new disk, initializing [ 411.786901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 411.837890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 411.865149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 416.668363] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 426.093292] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 430.891526] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 436.684623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 436.896350] Lustre: lustre-OST0000: new disk, initializing [ 436.900400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 436.907304] Lustre: 17042:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 436.991444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 438.345167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 438.357992] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 438.459302] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 442.252406] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 451.092665] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 455.885274] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 456.070778] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 460.626215] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 460.801133] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 464.355452] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 464.356962] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 464.368650] LustreError: Skipped 1 previous similar message [ 464.379946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 464.393025] Lustre: Skipped 4 previous similar messages [ 468.841353] Lustre: server umount lustre-OST0000 complete [ 473.057401] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 473.072928] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 473.088679] Lustre: Skipped 1 previous similar message [ 473.096130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 484.832575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 484.838108] Lustre: Skipped 5 previous similar messages [ 486.368988] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 486.561135] Lustre: server umount lustre-MDT0000 complete [ 489.819237] LustreError: 15063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787192 with bad export cookie 7121502201919851157 [ 489.820286] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 489.831547] LustreError: 15063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 490.154979] Lustre: server umount lustre-MDT0001 complete [ 494.877384] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:33:16 (1787787196) [ 500.500578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 501.024175] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.161141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 505.286516] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 506.342607] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.457053] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.199079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.304703] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 524.001367] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 529.662277] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 536.841243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.803782] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 549.453839] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 553.037501] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 559.583957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 559.592700] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 559.608428] Lustre: Skipped 1 previous similar message [ 559.618465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 564.275856] Lustre: server umount lustre-MDT0000 complete [ 567.270768] LustreError: 20037:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 567.291933] LustreError: 20037:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 567.582252] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787270 with bad export cookie 7121502201919852039 [ 567.585162] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 567.591078] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 567.890154] Lustre: server umount lustre-MDT0001 complete [ 581.978952] Lustre: server umount lustre-OST0000 complete [ 600.510574] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 609.756966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 610.231024] LustreError: 23688:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 610.318980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 610.328106] Lustre: Skipped 2 previous similar messages [ 614.716447] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 622.986507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 627.430792] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 630.105883] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 636.006451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 641.898789] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 649.997757] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 650.180292] Lustre: lustre-OST0001: new disk, initializing [ 650.182646] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 650.188075] Lustre: 26005:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 652.037847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 652.052084] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 652.158688] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 656.531603] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 665.899995] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 669.644244] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 679.904863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 679.916537] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 679.941774] Lustre: Skipped 2 previous similar messages [ 679.949121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 679.956137] Lustre: Skipped 3 previous similar messages [ 685.382256] Lustre: server umount lustre-MDT0000 complete [ 686.051455] LustreError: 24418:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 686.070043] LustreError: 24418:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 688.666704] LustreError: 26975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787391 with bad export cookie 7121502201919852879 [ 688.673481] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 688.679514] LustreError: 26975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 688.938525] Lustre: server umount lustre-MDT0001 complete [ 702.779207] Lustre: server umount lustre-OST0000 complete [ 715.314628] Lustre: server umount lustre-OST0001 complete [ 729.729125] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 737.544633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 741.019910] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 748.465711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 748.880067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 748.885709] Lustre: Skipped 4 previous similar messages [ 752.794683] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 755.611063] Lustre: 30122:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 761.316842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 762.742271] LustreError: 30565:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 762.766892] LustreError: 30565:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 767.261504] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 775.548163] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 780.742552] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 790.929562] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 795.470576] Lustre: 32002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 805.344621] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 805.359111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 805.376181] Lustre: Skipped 3 previous similar messages [ 805.381114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 805.385623] Lustre: Skipped 4 previous similar messages [ 808.372153] Lustre: server umount lustre-MDT0000 complete [ 811.959901] LustreError: 28957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787514 with bad export cookie 7121502201919854174 [ 811.966720] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 811.972837] LustreError: 28957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 812.201416] Lustre: server umount lustre-MDT0001 complete [ 825.922620] Lustre: server umount lustre-OST0000 complete [ 838.284949] Lustre: server umount lustre-OST0001 complete [ 846.353176] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:39:07 (1787787547) [ 873.441199] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 881.882265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 886.112414] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 892.387140] LustreError: 34805:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 892.414604] LustreError: 34805:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 893.720043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 898.092185] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 900.859184] Lustre: 35953:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 908.132948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 914.272927] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 923.381086] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 930.174945] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 941.330894] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 946.043767] Lustre: 37831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 953.830851] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 953.834642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 953.846554] Lustre: Skipped 5 previous similar messages [ 953.855458] Lustre: Skipped 7 previous similar messages [ 959.242649] Lustre: server umount lustre-MDT0000 complete [ 962.909125] LustreError: 34785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787665 with bad export cookie 7121502201919855287 [ 962.917488] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 962.932880] LustreError: 34785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 963.251865] Lustre: server umount lustre-MDT0001 complete [ 977.799390] Lustre: server umount lustre-OST0000 complete [ 980.960072] Lustre: 3641:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787787667/real 1787787667] req@ffff8e71753f03c0 x1874630343467904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787787683 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 981.190142] Lustre: server umount lustre-OST0001 complete [ 989.488737] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:41:30 (1787787690) [ 1005.455620] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1015.464604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1015.954549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1015.964613] Lustre: Skipped 6 previous similar messages [ 1020.432761] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1029.015949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1033.802837] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1037.126239] Lustre: 41126:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1044.972775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1052.268963] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1061.720696] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1069.742503] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1077.910438] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1081.386843] Lustre: 43004:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1082.624814] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1082.635371] Lustre: Skipped 1 previous similar message [ 1085.926813] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1085.942818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1085.963219] Lustre: Skipped 2 previous similar messages [ 1092.144415] Lustre: server umount lustre-MDT0000 complete [ 1096.736942] LustreError: 39957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787799 with bad export cookie 7121502201919856393 [ 1096.740835] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1096.754864] LustreError: 39957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1103.381932] Lustre: server umount lustre-MDT0001 complete [ 1112.638446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1121.766452] LustreError: 3640:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8e7170603480 x1874630343520768/t0(0) o250->MGC192.168.206.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1127.456838] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1131.824386] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1138.035467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1143.350868] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1143.793494] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1143.809537] Lustre: Skipped 1 previous similar message [ 1151.360636] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1152.650827] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1158.649788] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1160.232750] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1169.389351] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1171.582783] Lustre: server umount lustre-MDT0000 complete [ 1174.502393] LustreError: 44626:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1174.521650] LustreError: 44626:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 1175.059944] LustreError: 39957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787787878 with bad export cookie 7121502201919857527 [ 1175.065731] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1175.069525] LustreError: 39957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1175.444652] Lustre: server umount lustre-MDT0001 complete [ 1188.941668] Lustre: server umount lustre-OST0000 complete [ 1202.602333] Lustre: server umount lustre-OST0001 complete [ 1212.625614] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:45:13 (1787787913) [ 1230.312627] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1240.973528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1246.575838] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1255.151108] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1260.270507] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1263.252114] Lustre: 48766:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1270.937203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1277.437963] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1287.611661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1295.088192] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1302.911578] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1306.830532] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1318.887120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1318.904153] Lustre: Skipped 21 previous similar messages [ 1325.024496] Lustre: server umount lustre-MDT0000 complete [ 1329.050662] LustreError: 47593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787788032 with bad export cookie 7121502201919858325 [ 1329.052823] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1329.059326] LustreError: 47593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1329.674289] Lustre: server umount lustre-MDT0001 complete [ 1335.878448] Lustre: server umount lustre-OST0000 complete [ 1339.859237] Lustre: server umount lustre-OST0001 complete [ 1349.305915] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:47:30 (1787788050) [ 1351.879732] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1353.728081] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:47:34 (1787788054) [ 1363.826797] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 1373.941894] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1395.923470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1396.137985] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1396.247370] Lustre: lustre-MDT0000: new disk, initializing [ 1396.383248] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1400.377481] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1409.671373] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1416.305448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1416.574308] Lustre: lustre-OST0000: new disk, initializing [ 1416.581484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1416.593254] Lustre: Skipped 1 previous similar message [ 1416.598503] Lustre: 54841:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1417.733549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1417.746365] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1417.780374] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1422.228368] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1430.223117] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1432.805703] Lustre: DEBUG MARKER: create test files [ 1468.903968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1468.929091] Lustre: Skipped 15 previous similar messages [ 1468.939562] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1471.439101] Lustre: server umount lustre-OST0000 complete [ 1476.259835] Lustre: server umount lustre-MDT0000 complete [ 1488.242713] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 1491.135485] Key type lgssc unregistered [ 1491.472665] LNet: 56473:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1491.491348] LNetError: 56473:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1491.520212] LNet: Removed LNI 192.168.206.144@tcp [ 1492.264146] Key type .llcrypt unregistered [ 1492.270615] Key type ._llcrypt unregistered [ 1518.218734] Key type ._llcrypt registered [ 1518.221342] Key type .llcrypt registered [ 1518.343795] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1519.271072] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1519.317175] alg: No test for adler32 (adler32-zlib) [ 1520.470516] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 1520.707247] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 1522.343152] Key type lgssc registered [ 1523.284362] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1529.658301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1529.674424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1531.009586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1535.499580] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1543.038960] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1549.197201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1549.531544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1555.688807] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1555.794572] Lustre: 59189:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1555.806254] Lustre: 59189:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1555.816460] Lustre: 59189:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1555.822274] Lustre: 59189:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1555.827181] Lustre: 59189:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1555.831060] Lustre: 59189:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1556.033339] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1563.046513] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1568.201023] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1573.878810] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1581.536942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1581.546190] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1581.561238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1586.668760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1591.780625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1595.360432] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1595.503927] Lustre: server umount lustre-OST0000 complete [ 1599.151359] Lustre: server umount lustre-MDT0000 complete [ 1610.100334] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 1612.964736] Key type lgssc unregistered [ 1613.272877] LNet: 60802:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1613.287543] LNetError: 60802:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1613.314570] LNet: Removed LNI 192.168.206.144@tcp [ 1614.123527] Key type .llcrypt unregistered [ 1614.125333] Key type ._llcrypt unregistered [ 1644.868623] Key type ._llcrypt registered [ 1644.872298] Key type .llcrypt registered [ 1644.960953] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1646.251389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1646.276936] alg: No test for adler32 (adler32-zlib) [ 1647.279856] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 1647.537417] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 1649.183159] Key type lgssc registered [ 1650.267714] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1657.505987] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1657.522471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1659.008625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1663.024492] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1670.229771] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1676.552798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1676.837691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1682.849808] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1684.015376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1684.096553] Lustre: 63645:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1684.108631] Lustre: 63645:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1684.116842] Lustre: 63645:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1684.126158] Lustre: 63645:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1684.132974] Lustre: 63645:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1684.143621] Lustre: 63645:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1690.098191] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1695.063687] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1704.418260] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1704.420759] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1704.432898] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1709.536913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1710.152523] Lustre: server umount lustre-OST0000 complete [ 1713.746766] Lustre: server umount lustre-MDT0000 complete [ 1723.116040] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 1725.220375] Key type lgssc unregistered [ 1725.455807] LNet: 65208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1725.464873] LNetError: 65208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1725.487374] LNet: Removed LNI 192.168.206.144@tcp [ 1726.142819] Key type .llcrypt unregistered [ 1726.146091] Key type ._llcrypt unregistered [ 1742.466589] Key type ._llcrypt registered [ 1742.469375] Key type .llcrypt registered [ 1742.567411] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 1756.656493] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1757.789634] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1757.827504] alg: No test for adler32 (adler32-zlib) [ 1758.880987] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 1759.090794] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 1760.711162] Key type lgssc registered [ 1761.792130] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1796.958838] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1797.026216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1798.296308] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1798.319925] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1798.414867] Lustre: lustre-MDT0000: new disk, initializing [ 1798.465139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1798.476233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1802.575347] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1814.152708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1814.274567] Lustre: 68371:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1814.331190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1814.338532] Lustre: Skipped 1 previous similar message [ 1814.408187] Lustre: lustre-MDT0001: new disk, initializing [ 1814.494552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1814.534877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1814.546959] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1819.059647] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1830.290322] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1836.024958] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1843.188080] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1843.458861] Lustre: lustre-OST0000: new disk, initializing [ 1843.461575] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1843.467593] Lustre: 70337:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1843.564505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1848.874820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1848.885690] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1848.978431] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1850.066977] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 1859.595347] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1863.980875] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1864.159693] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1869.547349] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1869.851464] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1874.402758] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1874.417561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1874.433585] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1880.032840] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1880.047608] Lustre: Skipped 2 previous similar messages [ 1885.155105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1885.162802] Lustre: Skipped 1 previous similar message [ 1886.175252] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1886.340616] Lustre: server umount lustre-OST0000 complete [ 1890.273845] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1890.275470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1890.292856] Lustre: Skipped 2 previous similar messages [ 1890.307184] Lustre: Skipped 1 previous similar message [ 1895.396291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1895.405138] Lustre: Skipped 1 previous similar message [ 1896.162143] Lustre: server umount lustre-MDT0000 complete [ 1900.309366] LustreError: 68363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787788603 with bad export cookie 2361792452740983765 [ 1900.311483] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1900.321550] LustreError: 68363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1900.779403] Lustre: server umount lustre-MDT0001 complete [ 1911.643592] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:56:52 (1787788612) [ 1924.491581] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 1964.654845] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 1975.007174] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 2012.798617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2013.048086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2013.073662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2013.137279] Lustre: lustre-MDT0000: new disk, initializing [ 2013.211557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2013.234893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2017.454490] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2027.401550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2027.514816] Lustre: 76370:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2027.600859] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2027.608544] Lustre: Skipped 1 previous similar message [ 2027.705672] Lustre: lustre-MDT0001: new disk, initializing [ 2027.831594] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2027.870307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2027.879564] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2032.295486] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2041.719862] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2048.846461] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2055.973428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2056.309757] Lustre: lustre-OST0000: new disk, initializing [ 2056.320153] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2056.335282] Lustre: 78336:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2056.480424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2062.338385] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2062.894731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2062.909572] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2063.036109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2072.217497] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2076.851603] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2077.106608] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2081.381705] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2081.632136] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2083.812405] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2083.818450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2083.821523] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2083.831973] Lustre: Skipped 1 previous similar message [ 2089.733091] Lustre: server umount lustre-OST0000 complete [ 2094.054166] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2094.063881] Lustre: Skipped 1 previous similar message [ 2099.725531] Lustre: server umount lustre-MDT0000 complete [ 2103.451117] LustreError: 76362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787788806 with bad export cookie 2361792452740984647 [ 2103.453411] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2103.456580] LustreError: 76362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2103.744739] Lustre: server umount lustre-MDT0001 complete [ 2111.496140] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:00:12 (1787788812) [ 2117.557991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2118.073273] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2121.825333] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2123.241665] LustreError: 80750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2123.269946] LustreError: 80750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2127.330378] LustreError: 80749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2129.446544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2133.193809] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2140.188104] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2144.527113] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2149.276408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2154.706882] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2161.984440] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2165.732478] Lustre: 83433:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 6b1cb2d9-a3fd-4df6-a84e-726c14a61c12 at adminstrative request [ 2168.294441] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2168.310927] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2178.536359] Lustre: MGS: Client 6b1cb2d9-a3fd-4df6-a84e-726c14a61c12 (at 0@lo) reconnecting [ 2178.543979] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x20c6c5b8f2d332b9 to 0x20c6c5b8f2d33559 [ 2178.554574] Lustre: MGC192.168.206.144@tcp: Connection restored to 0@lo (at 0@lo) [ 2202.284073] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2202.430527] Lustre: lustre-OST0001: new disk, initializing [ 2202.434959] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2202.442361] Lustre: 83826:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2202.515754] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2202.519841] Lustre: Skipped 2 previous similar messages [ 2207.998642] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2208.903622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2208.922504] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2209.076583] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2216.242254] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2219.494203] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2219.497252] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2219.502628] LustreError: Skipped 1 previous similar message [ 2219.504302] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2219.504310] Lustre: Skipped 7 previous similar messages [ 2219.535764] Lustre: Skipped 1 previous similar message [ 2224.875893] Lustre: server umount lustre-OST0001 complete [ 2229.731116] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2229.734359] LustreError: 83270:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2229.736660] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2229.741205] LustreError: Skipped 1 previous similar message [ 2229.768103] LustreError: 83270:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2234.187308] Lustre: server umount lustre-OST0000 complete [ 2237.408263] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2243.663312] Lustre: server umount lustre-MDT0000 complete [ 2245.090979] LustreError: 81476:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2245.115556] LustreError: 81476:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2247.320548] LustreError: 80729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787788950 with bad export cookie 2361792452740986201 [ 2247.330447] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2247.338511] LustreError: 80729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2247.736368] Lustre: server umount lustre-MDT0001 complete [ 2256.920311] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:02:38 (1787788958) [ 2258.347206] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2260.101513] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:02:41 (1787788961) [ 2261.515529] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2263.376510] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:02:44 (1787788964) [ 2264.984979] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2266.948574] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:02:48 (1787788968) [ 2268.530318] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2270.391596] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:02:51 (1787788971) [ 2272.145831] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2277.385037] loop: module loaded [ 2277.393144] loop0: detected capacity change from 0 to 4294967296 [ 2277.446718] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2278.940532] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2282.537254] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2286.376475] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2308.878786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2308.922871] LustreError: 87947:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2308.927717] LustreError: 87947:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2308.933072] LustreError: 87947:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2308.943115] LustreError: 87947:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2327.625621] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 2336.817157] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 2367.856555] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2368.081691] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2368.103639] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2368.175435] Lustre: lustre-MDT0000: new disk, initializing [ 2368.258568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2368.278743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2372.562061] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2381.837718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2381.987550] Lustre: 90809:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2382.184786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2385.514325] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2392.736481] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2397.063838] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2402.279750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2402.482318] Lustre: 92782:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2406.955777] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2411.546967] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2411.635745] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2414.457368] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2418.770868] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2418.994796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2422.835890] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2423.012357] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2426.848304] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2426.853606] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2426.863481] Lustre: Skipped 3 previous similar messages [ 2426.866749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2426.869759] Lustre: Skipped 8 previous similar messages [ 2430.617417] Lustre: server umount lustre-OST0000 complete [ 2439.488308] Lustre: server umount lustre-MDT0000 complete [ 2442.211895] LustreError: 92036:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2442.235480] LustreError: 92036:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2442.576270] LustreError: 90804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787789145 with bad export cookie 2361792452740986817 [ 2442.577939] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2442.594887] LustreError: 90804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2442.786968] Lustre: server umount lustre-MDT0001 complete [ 2450.153704] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:05:51 (1787789151) [ 2455.691891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2456.093126] LustreError: 95194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2459.542245] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2465.940277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2469.778771] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2475.818475] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2480.596679] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2486.356314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2491.371880] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2498.109564] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2502.409878] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2503.688337] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 2507.804683] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2508.004651] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2513.791796] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2513.928405] Lustre: lustre-OST0001: new disk, initializing [ 2513.930422] Lustre: Skipped 2 previous similar messages [ 2513.932766] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2513.946118] Lustre: Skipped 3 previous similar messages [ 2513.953583] Lustre: 98568:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2514.014264] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2514.022257] Lustre: Skipped 5 previous similar messages [ 2520.218342] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2524.242571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2524.249098] Lustre: Skipped 2 previous similar messages [ 2524.258139] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2524.349357] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2528.702319] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2533.175890] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2533.450241] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2537.322781] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2537.502943] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2554.230180] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2576.791493] Lustre: Failing over lustre-MDT0000 [ 2577.132056] Lustre: server umount lustre-MDT0000 complete [ 2578.913222] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2578.923074] LustreError: Skipped 1 previous similar message [ 2578.927698] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2578.941504] Lustre: Skipped 3 previous similar messages [ 2578.946292] LustreError: 95194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2578.960667] LustreError: 95194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2596.255261] Lustre: 66593:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787789283/real 1787789283] req@0000000050f30a40 x1874632074591232/t0(0) o400->MGC192.168.206.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787789299 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2596.285783] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2599.197597] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2599.204441] LDISKFS-fs (dm-0): recovery complete [ 2599.226034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2606.561884] LustreError: 66590:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000009b93070e x1874632074598144/t0(0) o250->MGC192.168.206.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2606.848780] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2609.849247] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2610.990566] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2610.991589] Lustre: 95191:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2612.196866] Lustre: 95194:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2612.204146] Lustre: 95194:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2612.222928] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2612.270862] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2612.591592] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2612.914512] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2612.933471] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2613.255701] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2613.260908] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2613.267215] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2613.273684] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2614.217915] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2614.224815] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 2614.543681] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2614.559752] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 2615.512751] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2615.521362] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2616.464201] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2616.472834] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2616.791328] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2616.802030] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2619.666149] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2619.685780] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2620.642228] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2620.656294] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2620.983127] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2620.988288] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2627.920313] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2627.930739] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2628.898938] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2628.907841] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2629.231199] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2629.245743] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2644.097754] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2644.116423] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2645.073036] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2645.082683] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2645.399159] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2645.409970] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2670.047510] Lustre: 101421:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2670.062586] Lustre: 101421:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2670.071959] Lustre: 101421:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 13861c79-bbe1-45e8-8326-456b838e221b@192.168.206.44@tcp [ 2670.086423] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2670.095901] Lustre: 101421:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000f8dec6c3 x1874632058671360/t0(8589934779) o101->13861c79-bbe1-45e8-8326-456b838e221b@192.168.206.44@tcp:723/0 lens 592/0 e 0 to 0 dl 1787789383 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2670.116955] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2670.134556] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2670.137041] Lustre: lustre-MDT0000: Denying connection for new client 13861c79-bbe1-45e8-8326-456b838e221b (at 192.168.206.44@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2670.176545] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2670.183268] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2670.184491] Lustre: Skipped 2 previous similar messages [ 2670.217480] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2670.217509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2680.805067] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2680.819363] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2680.827027] Lustre: Skipped 3 previous similar messages [ 2680.831426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2680.834951] Lustre: Skipped 3 previous similar messages [ 2685.651594] Lustre: server umount lustre-OST0000 complete [ 2688.996310] LustreError: 98850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2689.010903] LustreError: 98850:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 2694.445806] Lustre: server umount lustre-OST0001 complete [ 2703.521302] Lustre: server umount lustre-MDT0000 complete [ 2706.496547] LustreError: 98567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787789409 with bad export cookie 2361792452741038036 [ 2706.500297] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2706.505950] LustreError: 98567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2706.769377] Lustre: server umount lustre-MDT0001 complete [ 2713.020256] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:10:14 (1787789414) [ 2719.173250] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2719.177676] Lustre: *** cfs_fail_loc=197, val=0*** [ 2734.688274] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2739.807454] LustreError: 103395:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.144@tcp: failed processing log, type 4: rc = -110 [ 2744.946092] Lustre: *** cfs_fail_loc=197, val=0*** [ 2744.949430] Lustre: Skipped 65 previous similar messages [ 2770.164054] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2775.861799] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2777.433274] Lustre: server umount lustre-OST0000 complete [ 2783.194987] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:11:24 (1787789484) [ 2801.983452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2802.386786] LustreError: 105015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2802.406766] LustreError: 105015:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2802.457344] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2802.465090] Lustre: Skipped 2 previous similar messages [ 2806.159494] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2814.710981] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2818.014984] LustreError: 105046:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2818.024426] LustreError: 105046:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2818.032753] LustreError: 105046:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2818.335789] Lustre: server umount lustre-MDT0000 complete [ 2850.092461] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 2858.845990] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 2885.002318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2885.231429] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2885.261381] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2885.333482] Lustre: lustre-MDT0000: new disk, initializing [ 2885.521365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2889.090523] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2897.903814] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2897.992515] Lustre: 109215:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2898.231947] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2902.369908] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2911.060485] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2916.229338] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2922.101433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2922.390147] Lustre: 111184:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2929.360201] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 2929.923593] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2940.040803] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2944.758234] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2944.949787] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2948.691085] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2948.871348] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2950.631761] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2950.634825] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2950.639451] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2950.639459] Lustre: Skipped 9 previous similar messages [ 2950.645115] LustreError: Skipped 2 previous similar messages [ 2950.668352] Lustre: Skipped 6 previous similar messages [ 2956.705162] Lustre: server umount lustre-OST0000 complete [ 2966.252362] Lustre: server umount lustre-MDT0000 complete [ 2969.414965] LustreError: 110012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787789672 with bad export cookie 2361792452741048711 [ 2969.420723] LustreError: 110012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2969.420798] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2969.882797] Lustre: server umount lustre-MDT0001 complete [ 2976.521612] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:14:38 (1787789678) [ 2983.025970] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 2985.133396] Key type lgssc unregistered [ 2985.448236] LNet: 113672:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2985.457693] LNetError: 113672:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2985.488358] LNet: Removed LNI 192.168.206.144@tcp [ 2986.190695] Key type .llcrypt unregistered [ 2986.195654] Key type ._llcrypt unregistered [ 2999.826745] Key type ._llcrypt registered [ 2999.829387] Key type .llcrypt registered [ 2999.893398] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 3011.750257] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3012.576681] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3012.604991] alg: No test for adler32 (adler32-zlib) [ 3013.614477] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 3013.768059] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 3015.407178] Key type lgssc registered [ 3016.251776] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3049.636991] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3049.656953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3050.870624] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3050.920499] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3051.014268] Lustre: lustre-MDT0000: new disk, initializing [ 3051.095911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3051.119488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3054.592371] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3062.720854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3063.032589] Lustre: lustre-OST0000: new disk, initializing [ 3063.036739] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3063.041956] Lustre: Skipped 1 previous similar message [ 3063.055923] Lustre: 117840:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3063.132201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3067.875777] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3071.063488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3071.072438] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3071.106263] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3076.176927] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3076.307412] Lustre: lustre-OST0001: new disk, initializing [ 3076.310983] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3076.316872] Lustre: 118702:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3076.370325] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3079.748856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3079.757842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3079.802256] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3081.683065] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3089.888902] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3089.892180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3089.903029] Lustre: Skipped 1 previous similar message [ 3089.914910] Lustre: Skipped 1 previous similar message [ 3095.008821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3095.019641] Lustre: Skipped 1 previous similar message [ 3096.200159] Lustre: server umount lustre-MDT0000 complete [ 3101.657774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3105.516781] LustreError: 117029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787789808 with bad export cookie 8307264993239619164 [ 3105.528791] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3105.629242] Lustre: server umount lustre-OST0000 complete [ 3108.729052] Lustre: server umount lustre-OST0001 complete [ 3114.927815] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 20:16:56 (1787789816) [ 3132.592810] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 20:17:14 (1787789834) [ 3140.240633] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3158.143976] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:17:39 (1787789859) [ 3189.617778] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 3201.001181] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3238.012612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3238.217660] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3238.258155] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3238.340247] Lustre: lustre-MDT0000: new disk, initializing [ 3238.398296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3238.414990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3242.340843] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3252.317155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3252.406819] Lustre: 126451:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3252.527108] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3252.534263] Lustre: Skipped 1 previous similar message [ 3252.629285] Lustre: lustre-MDT0001: new disk, initializing [ 3252.747426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3252.822975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3252.840967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3256.869303] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3265.209120] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3270.504842] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3277.176304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3277.403451] Lustre: lustre-OST0000: new disk, initializing [ 3277.407418] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3277.421653] Lustre: 128419:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3277.523035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3283.604445] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3286.569838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3286.594728] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3286.695375] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3292.765765] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3297.115661] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3297.318523] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3301.433953] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3301.591664] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3307.495404] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3307.499635] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3307.504434] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3307.515428] Lustre: Skipped 1 previous similar message [ 3309.788102] Lustre: server umount lustre-OST0000 complete [ 3314.148492] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3314.155816] LustreError: Skipped 1 previous similar message [ 3314.160117] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3314.176461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3314.181724] Lustre: Skipped 1 previous similar message [ 3317.730047] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3319.487354] Lustre: server umount lustre-MDT0000 complete [ 3322.750947] LustreError: 126444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790025 with bad export cookie 8307264993239620389 [ 3322.752780] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3322.760391] LustreError: 126444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3323.051632] Lustre: server umount lustre-MDT0001 complete [ 3330.816752] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:20:32 (1787790032) [ 3336.585961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3336.984270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3340.843617] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3342.311628] LustreError: 130833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3342.340693] LustreError: 130833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3346.402727] LustreError: 130834:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3348.306048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3352.538864] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3359.105928] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3364.438889] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3370.364350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3370.623859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3370.629439] Lustre: Skipped 1 previous similar message [ 3376.767450] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3383.437428] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3387.676074] Lustre: *** cfs_fail_loc=159, val=0*** [ 3404.135287] Lustre: lustre-MDT0000: Client 0b1fab12-2bf1-4cc4-b3a4-7d6fda161c39 (at 192.168.206.44@tcp) reconnecting [ 3405.692548] Lustre: *** cfs_fail_loc=159, val=0*** [ 3405.696366] Lustre: Skipped 5 previous similar messages [ 3422.031849] Lustre: lustre-MDT0000: Client 0b1fab12-2bf1-4cc4-b3a4-7d6fda161c39 (at 192.168.206.44@tcp) reconnecting [ 3438.563038] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3438.571441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3438.577382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3438.588806] Lustre: Skipped 1 previous similar message [ 3438.597458] Lustre: Skipped 3 previous similar messages [ 3444.394638] Lustre: server umount lustre-OST0000 complete [ 3448.801095] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3448.803744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3448.812568] Lustre: Skipped 1 previous similar message [ 3448.829374] Lustre: Skipped 3 previous similar messages [ 3462.112086] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3462.316987] Lustre: server umount lustre-MDT0000 complete [ 3464.162377] LustreError: 130833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3464.186922] LustreError: 130833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3465.222981] LustreError: 133552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790168 with bad export cookie 8307264993239621271 [ 3465.228873] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3465.235238] LustreError: 133552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3465.515782] Lustre: server umount lustre-MDT0001 complete [ 3473.864105] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 3476.007492] Key type lgssc unregistered [ 3476.257677] LNet: 134612:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3476.270331] LNetError: 134612:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3476.284691] LNet: Removed LNI 192.168.206.144@tcp [ 3476.977653] Key type .llcrypt unregistered [ 3476.980801] Key type ._llcrypt unregistered [ 3498.525364] Key type ._llcrypt registered [ 3498.527077] Key type .llcrypt registered [ 3498.654841] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3499.435158] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3499.468663] alg: No test for adler32 (adler32-zlib) [ 3500.618453] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 3500.864192] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 3502.559178] Key type lgssc registered [ 3504.084291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3513.591805] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3513.626129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3515.312256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3519.635969] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3525.603025] LustreError: 136027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3525.614908] LustreError: 136027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3527.915610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3528.247766] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3532.322546] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3538.665508] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3543.017814] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3548.727479] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3549.064149] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3554.324977] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3560.373780] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3561.498587] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3570.774477] Lustre: *** cfs_fail_loc=159, val=0*** [ 3587.413578] Lustre: lustre-MDT0000: Client 6fbb6395-67c8-48e4-8d73-6650b398bd6a (at 192.168.206.44@tcp) reconnecting [ 3589.344725] Lustre: *** cfs_fail_loc=159, val=0*** [ 3604.823259] Lustre: lustre-MDT0001: Client 6fbb6395-67c8-48e4-8d73-6650b398bd6a (at 192.168.206.44@tcp) reconnecting [ 3606.225752] Lustre: *** cfs_fail_loc=159, val=0*** [ 3606.230158] Lustre: Skipped 3 previous similar messages [ 3621.713222] Lustre: lustre-MDT0001: Client 6fbb6395-67c8-48e4-8d73-6650b398bd6a (at 192.168.206.44@tcp) reconnecting [ 3625.341830] Lustre: 139064:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3628.147886] Lustre: *** cfs_fail_loc=159, val=0*** [ 3628.150936] Lustre: Skipped 4 previous similar messages [ 3643.727233] Lustre: lustre-MDT0000: Client bc6ab892-f8ee-4844-9f9e-211a2ff43e3b (at 192.168.206.44@tcp) reconnecting [ 3645.056835] Lustre: *** cfs_fail_loc=159, val=0*** [ 3645.061071] Lustre: Skipped 13 previous similar messages [ 3660.626058] Lustre: lustre-MDT0000: Client bc6ab892-f8ee-4844-9f9e-211a2ff43e3b (at 192.168.206.44@tcp) reconnecting [ 3661.853394] Lustre: 139356:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3667.949201] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3667.951409] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3667.955240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3667.956900] LustreError: Skipped 1 previous similar message [ 3667.967659] Lustre: Skipped 1 previous similar message [ 3672.279603] Lustre: server umount lustre-OST0000 complete [ 3676.640928] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3676.647819] LustreError: Skipped 1 previous similar message [ 3676.651391] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3676.662312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3676.667239] Lustre: Skipped 1 previous similar message [ 3678.183112] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3678.183301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3678.212751] Lustre: Skipped 1 previous similar message [ 3681.437843] Lustre: server umount lustre-MDT0000 complete [ 3683.299096] LustreError: 139183:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3683.314913] LustreError: 139183:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3684.139188] LustreError: 136009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790387 with bad export cookie 6437644097253607715 [ 3684.142790] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3684.153484] LustreError: 136009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3684.420675] Lustre: server umount lustre-MDT0001 complete [ 3692.716783] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 3694.864462] Key type lgssc unregistered [ 3695.079757] LNet: 140408:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3695.086547] LNetError: 140408:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3695.098255] LNet: Removed LNI 192.168.206.144@tcp [ 3695.799907] Key type .llcrypt unregistered [ 3695.802833] Key type ._llcrypt unregistered [ 3714.862600] Key type ._llcrypt registered [ 3714.864259] Key type .llcrypt registered [ 3714.949922] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3715.741691] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3715.841713] alg: No test for adler32 (adler32-zlib) [ 3716.861111] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 3717.103751] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 3718.791166] Key type lgssc registered [ 3719.590532] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3725.475352] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3725.486779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3726.797299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3729.970591] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3736.036047] LustreError: 141801:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3736.048947] LustreError: 141801:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3736.238080] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3736.513974] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3739.430848] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3744.654648] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3748.590745] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3753.431328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3753.707752] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3759.103881] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3761.927427] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3765.786975] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3771.140123] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3776.172143] Lustre: 144384:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3778.547817] Lustre: 144435:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3779.730529] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3780.469970] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3783.483247] Lustre: 144582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3784.195600] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3784.798865] Lustre: 144680:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3791.839994] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3791.849819] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3791.856080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3791.867870] Lustre: Skipped 1 previous similar message [ 3794.424517] Lustre: server umount lustre-OST0000 complete [ 3797.984380] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3797.993422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3798.021099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3798.030153] Lustre: Skipped 1 previous similar message [ 3802.080311] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.082838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3802.105845] Lustre: Skipped 1 previous similar message [ 3804.039160] Lustre: server umount lustre-MDT0000 complete [ 3807.142328] LustreError: 142535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790510 with bad export cookie 13567595392823422522 [ 3807.145220] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3807.155258] LustreError: 142535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3807.444945] Lustre: server umount lustre-MDT0001 complete [ 3816.513978] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 3818.932465] Key type lgssc unregistered [ 3819.230730] LNet: 145744:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3819.238614] LNetError: 145744:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3819.259174] LNet: Removed LNI 192.168.206.144@tcp [ 3820.101936] Key type .llcrypt unregistered [ 3820.105447] Key type ._llcrypt unregistered [ 3841.973836] Key type ._llcrypt registered [ 3841.977872] Key type .llcrypt registered [ 3842.124422] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3843.139106] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3843.312723] alg: No test for adler32 (adler32-zlib) [ 3844.442318] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 3844.628122] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 3846.271190] Key type lgssc registered [ 3847.102216] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3853.432955] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3853.465177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3854.890539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3858.745297] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3864.041599] LustreError: 147160:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3864.060519] LustreError: 147160:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3866.126802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3866.443387] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3870.246652] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3876.420988] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3881.114438] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3886.950584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3887.288436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3891.376721] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3892.557859] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 3897.818758] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3899.565913] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3907.446750] Lustre: *** cfs_fail_loc=159, val=0*** [ 3923.840171] Lustre: lustre-MDT0000: Client 77a3c11a-62b9-4fd2-9114-c2f662c6066c (at 192.168.206.44@tcp) reconnecting [ 3927.520614] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3927.525793] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3927.534553] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3930.091136] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3930.098981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3930.100866] LustreError: Skipped 1 previous similar message [ 3930.114673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3933.499538] Lustre: server umount lustre-OST0000 complete [ 3938.275935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3938.286486] LustreError: Skipped 4 previous similar messages [ 3938.291676] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3938.305388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3938.311952] Lustre: Skipped 1 previous similar message [ 3940.834761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3940.836120] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3942.984361] Lustre: server umount lustre-MDT0000 complete [ 3945.730633] LustreError: 147140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790648 with bad export cookie 5300047896075751539 [ 3945.736365] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3945.740023] LustreError: 147140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3945.966511] Lustre: server umount lustre-MDT0001 complete [ 3954.243495] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 3956.678940] Key type lgssc unregistered [ 3956.957592] LNet: 150823:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3956.965243] LNetError: 150823:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3956.979066] LNet: Removed LNI 192.168.206.144@tcp [ 3957.829260] Key type .llcrypt unregistered [ 3957.831489] Key type ._llcrypt unregistered [ 3982.259511] Key type ._llcrypt registered [ 3982.265206] Key type .llcrypt registered [ 3982.400805] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 3983.631138] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3983.669580] alg: No test for adler32 (adler32-zlib) [ 3984.691307] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 3984.964618] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 3986.679152] Key type lgssc registered [ 3987.660893] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3994.073288] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3994.101981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3995.504563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3999.487592] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4005.856512] LustreError: 152239:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4005.871935] LustreError: 152239:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4006.468091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4006.827339] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4010.857836] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4017.167641] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4022.030942] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4028.050222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4028.324294] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4031.433239] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4033.449848] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4039.248059] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4040.679880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4057.626670] Lustre: lustre-MDT0001: evicting c3356913-efe5-4f8b-a689-a2dee7e6b762 (at 192.168.206.44@tcp) by administrative request [ 4057.634497] Lustre: Skipped 1 previous similar message [ 4086.757551] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4086.759961] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4086.762442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4086.767474] Lustre: Skipped 1 previous similar message [ 4090.244552] Lustre: server umount lustre-OST0000 complete [ 4093.921893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4093.932434] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4093.950896] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4093.956379] Lustre: Skipped 1 previous similar message [ 4096.994528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4096.995352] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4099.655481] Lustre: server umount lustre-MDT0000 complete [ 4102.116765] LustreError: 154737:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4102.136317] LustreError: 154737:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4102.674929] LustreError: 152221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790805 with bad export cookie 2412986451879594435 [ 4102.682739] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4102.693700] LustreError: 152221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4103.009626] Lustre: server umount lustre-MDT0001 complete [ 4111.344403] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 4113.570847] Key type lgssc unregistered [ 4113.808740] LNet: 156311:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4113.817322] LNetError: 156311:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4114.856439] LNet: Removed LNI 192.168.206.144@tcp [ 4115.724200] Key type .llcrypt unregistered [ 4115.726370] Key type ._llcrypt unregistered [ 4175.366520] Key type ._llcrypt registered [ 4175.368628] Key type .llcrypt registered [ 4175.451879] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 4176.132059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4176.221383] alg: No test for adler32 (adler32-zlib) [ 4177.258943] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 4177.544080] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 4179.192279] Key type lgssc registered [ 4180.248800] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4187.005307] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4187.024221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4188.427412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4191.944890] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4198.890632] LustreError: 158778:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4198.907784] LustreError: 158778:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4198.968383] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4199.303412] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4202.851814] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4208.991265] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4214.391293] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4219.685231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4219.973803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4224.114602] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4224.554654] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4230.321202] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4233.335926] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4243.939811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4243.940311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4243.957652] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4243.964828] Lustre: Skipped 1 previous similar message [ 4243.976760] Lustre: Skipped 1 previous similar message [ 4247.358714] Lustre: server umount lustre-OST0000 complete [ 4250.592237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4250.595952] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4250.602884] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4254.178741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4254.182440] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4256.583517] Lustre: server umount lustre-MDT0000 complete [ 4259.254086] LustreError: 158759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787790962 with bad export cookie 3347822905529724605 [ 4259.265655] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4259.269374] LustreError: 158759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4259.576251] Lustre: server umount lustre-MDT0001 complete [ 4268.511661] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 4270.759411] Key type lgssc unregistered [ 4271.004410] LNet: 162321:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4271.019550] LNetError: 162321:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4271.037933] LNet: Removed LNI 192.168.206.144@tcp [ 4271.659227] Key type .llcrypt unregistered [ 4271.661956] Key type ._llcrypt unregistered [ 4290.698051] Key type ._llcrypt registered [ 4290.703506] Key type .llcrypt registered [ 4290.833700] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 4291.581627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4291.601616] alg: No test for adler32 (adler32-zlib) [ 4292.716547] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 4293.018333] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 4294.679845] Key type lgssc registered [ 4295.951816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4308.830781] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:36:50 (1787791010) [ 4323.147737] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 4331.496204] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 4361.823835] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4361.850284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4363.082683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4363.132704] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4363.188693] Lustre: lustre-MDT0000: new disk, initializing [ 4363.254768] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4363.274202] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4366.932225] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4375.355367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4375.485871] Lustre: 166560:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4375.540496] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4375.544223] Lustre: Skipped 1 previous similar message [ 4375.604298] Lustre: lustre-MDT0001: new disk, initializing [ 4375.679591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4375.718683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4375.732119] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4379.371889] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4387.271620] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4391.326606] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4396.156919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4396.393539] Lustre: lustre-OST0000: new disk, initializing [ 4396.397665] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4396.402805] Lustre: 168529:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4396.470501] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4401.335216] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4404.757191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4404.768385] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4404.827768] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4408.184229] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4420.070892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4420.074531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4420.074648] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4420.080933] Lustre: Skipped 1 previous similar message [ 4420.106250] Lustre: Skipped 1 previous similar message [ 4422.220854] Lustre: server umount lustre-OST0000 complete [ 4425.190089] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4425.208549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4425.214209] Lustre: Skipped 1 previous similar message [ 4430.307032] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4430.317185] Lustre: Skipped 1 previous similar message [ 4431.341510] Lustre: server umount lustre-MDT0000 complete [ 4433.943281] LustreError: 168528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791136 with bad export cookie 13191012895663333705 [ 4433.952564] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4433.959696] LustreError: 168528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4434.242552] Lustre: server umount lustre-MDT0001 complete [ 4442.235161] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 4444.554776] Key type lgssc unregistered [ 4444.765748] LNet: 170436:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4444.771520] LNetError: 170436:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4444.783109] LNet: Removed LNI 192.168.206.144@tcp [ 4445.545855] Key type .llcrypt unregistered [ 4445.548374] Key type ._llcrypt unregistered [ 4464.262274] Key type ._llcrypt registered [ 4464.263770] Key type .llcrypt registered [ 4464.340580] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 4465.157354] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4465.176045] alg: No test for adler32 (adler32-zlib) [ 4466.170805] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 4466.371211] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 4468.040762] Key type lgssc registered [ 4468.888745] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4474.296213] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4474.313570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4475.632193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4478.531239] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4484.459932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4484.728224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4487.900739] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4492.986371] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4497.237901] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4502.142902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4502.405679] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4508.596694] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 4515.915325] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4521.904590] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4526.606657] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4526.631322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.642789] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4526.651057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.669037] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4527.455778] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4527.488929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4527.499982] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4527.516226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.534703] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4528.575613] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4528.591350] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4528.603209] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4528.609437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.620349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4529.432817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.249757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.173558] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4531.179459] Lustre: Skipped 2 previous similar messages [ 4531.190308] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4531.205438] Lustre: Skipped 2 previous similar messages [ 4531.211846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4531.215345] Lustre: Skipped 2 previous similar messages [ 4531.217457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.229979] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4531.238922] Lustre: Skipped 2 previous similar messages [ 4532.028599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.921906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.837319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.536768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.499434] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4535.508044] Lustre: Skipped 4 previous similar messages [ 4535.533496] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4535.543619] Lustre: Skipped 4 previous similar messages [ 4535.548027] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4535.553174] Lustre: Skipped 4 previous similar messages [ 4535.556895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.563581] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4535.568891] Lustre: Skipped 4 previous similar messages [ 4536.340713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.201064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.910331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.804147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.668204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.313203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.101023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.914709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.688868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.543472] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4543.553647] Lustre: Skipped 9 previous similar messages [ 4543.563802] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.572833] Lustre: Skipped 9 previous similar messages [ 4543.577410] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4543.585310] Lustre: Skipped 9 previous similar messages [ 4543.589927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.603905] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4543.610753] Lustre: Skipped 9 previous similar messages [ 4544.371476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.074242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.925755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.640637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.409211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.175294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.959460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.675690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.558064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.369351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.136194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.853398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.648967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.413582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.387374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.081662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.863606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.545704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.292212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.034842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.678436] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4559.687208] Lustre: Skipped 20 previous similar messages [ 4559.702666] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4559.710968] Lustre: Skipped 20 previous similar messages [ 4559.714878] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4559.718695] Lustre: Skipped 20 previous similar messages [ 4559.724687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.730511] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4559.735436] Lustre: Skipped 20 previous similar messages [ 4560.509621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.321098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.038766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.812183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.447710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.177354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.964231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.674803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.370979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.079214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.776180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.601203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.327393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.052546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.806945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.693360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.418695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.154845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.857179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.563531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.366190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.079668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.822731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.614764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.317220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.077248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.842706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.533240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.246196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.983626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.852131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.630449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.412861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.234482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.125028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.855796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.862843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.700491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.517508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.427917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.184041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.089542] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4592.098664] Lustre: Skipped 41 previous similar messages [ 4592.111880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4592.122890] Lustre: Skipped 41 previous similar messages [ 4592.126397] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4592.131828] Lustre: Skipped 41 previous similar messages [ 4592.134893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.144639] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4592.155939] Lustre: Skipped 41 previous similar messages [ 4593.013682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.830895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.694280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.430359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.147679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.904697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.785793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.552454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.368769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.133589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.021613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.980576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.826476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.736368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.556236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.286759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.162958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.108141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.978741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.718244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.576404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.417864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.222328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.064939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.893420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.644964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.467857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.227367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.048868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.890622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.654621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.444475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.259108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.070600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.915564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.785436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.719247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.581082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.443965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.354156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.202823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.073771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.892975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.788594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.645218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.451687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.370610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.180340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.231500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.154139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.118429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.022703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.864350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.553410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.324525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.004833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.884392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.656626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.502920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.180681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.896198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.610318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.424340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.273466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.029725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.868869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.932663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.829470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.583282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.435989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.297785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.062658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.857230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.627488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.611773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.429353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.391624] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4656.397348] Lustre: Skipped 76 previous similar messages [ 4656.419941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4656.433277] Lustre: Skipped 76 previous similar messages [ 4656.436610] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4656.440246] Lustre: Skipped 76 previous similar messages [ 4656.443516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.454330] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4656.467321] Lustre: Skipped 76 previous similar messages [ 4657.337629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.274455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.247851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.127715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.900943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.803380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.709579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.577500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.472346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.301970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.136569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.055567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.957676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.987220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.873229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.703413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.555611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.276451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.072353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.056112] LustreError: 171848:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4674.064424] LustreError: 171848:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4674.076517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.782602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.650530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.464460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.376863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.301562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.293086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.301105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.133827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.088424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.036600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.944538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.851503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.069246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.231979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.257585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.143044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.040502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.903253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.945116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.947517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.846216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.741716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.648655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.568888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.426917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.195301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.007285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.825313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.629791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.414590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.179973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.942668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.779767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.582693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.500432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.195449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.056400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.899813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.787425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.547274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.534075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.230476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.875966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.616654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.282500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.088029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.762555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.644369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.496490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.544383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.388437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.429152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.217915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.013413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.767566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.774490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.609378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.517556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.487551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.474572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.326164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.302818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.102107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.081403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.905804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.675113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.441732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.177924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.989756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.731237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.544775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.358514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.163436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.069537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.909803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.885372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.849673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.873829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.738041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.752192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.729080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.666622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.870870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.897496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.977668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.965627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.024775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.051905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.100367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.098988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.063801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.901448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.785597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.872329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.854606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.795935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.610824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.544605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.449978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.399829] LustreError: 171848:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4763.410045] LustreError: 171848:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4763.425877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.417892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.309321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.080993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.073248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.034495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.001490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.918644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.850339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.676348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.751943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.827554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.929119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.949841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.018945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.224426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.139092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.107886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.109665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.199110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.174228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.423250] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4784.434902] Lustre: Skipped 140 previous similar messages [ 4784.447074] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4784.459820] Lustre: Skipped 140 previous similar messages [ 4784.463499] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4784.474098] Lustre: Skipped 140 previous similar messages [ 4784.478647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.493092] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4784.503638] Lustre: Skipped 140 previous similar messages [ 4785.565086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.675284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.607756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.685447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.511293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.470231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.386596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.489285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.297642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.242572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.101454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.066916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.042642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.181304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.065476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.959460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.923493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.094708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.031316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.833318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.686850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.783257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.907519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.876160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.029156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.059024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.132633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.993693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.024489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.049943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.977590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.877649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.826483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.593794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.586835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.413986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.290494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.061737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.101498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.103935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.060140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.102225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.870203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.722201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.665187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.529073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.528543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.298620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.129323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.997411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.846958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.739786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.555168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.383359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.259741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.019310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.779145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.678699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.469389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.408924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.435837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.258712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.216239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.055398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.857239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.810549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.543506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.248902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.980378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.678653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.412877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.187886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.002174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.737066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.608795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.353503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.193094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.961518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.863284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.731560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.479109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.263812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.032482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.884644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.646947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.543729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.429082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.182255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.046064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.986963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.848104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.837251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.813677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.632326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.398790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.188890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.117434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.018814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.871297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.648785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.685512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.503392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.488959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.387359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.330816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.224288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.021512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.817423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.787391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.726870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.652312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.522249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.385989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.240978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.101623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.995520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.915706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.734762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.664074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.544309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.359948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.306304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.221921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.030080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.821748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.711494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.651751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.648938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.575067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.556045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.512109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.529061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.389903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.234773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.116263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.139563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.961672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.768599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.629261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.485905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.293585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.408573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.541490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.688382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.796912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.971851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.467701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.875221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.447197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.878229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.671222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.494932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.672927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.875317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.329796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.709734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.956216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.118025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.029322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.084447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.214978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.111201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.927270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.649299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.395083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.061598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.837992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.823567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.916539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.220362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.989176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.850873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.748082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.818809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.463451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.588600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.701507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.613805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.702987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.860990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.278892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.234799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.354501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.318550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.292861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.148579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.043743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.926690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.983300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.771104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.732223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.734207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.630931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.430558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.324689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.037034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.773949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.680787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.544200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.364923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.180589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.041231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.012652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.198537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.989761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.803398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.544764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.254618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.113134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.008105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.729583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.689820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.656486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.462454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.212937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.020400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.028979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.976553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.847066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.717753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.775947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.769443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.729365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.654459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.337869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.185912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.057079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.987759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.792589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.644268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.411616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.152189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.975201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.736516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.488855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.240804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.017960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.852288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.768934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.743652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.590533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.616792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.573301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.496478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.396545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.273178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.090253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.931301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.777440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.586666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.413818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.233182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.216973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.146537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.129624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.948393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.806352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.672782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.455874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.183401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.959335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.797927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.733415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.565683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.478698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.371672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.319114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.256083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.243919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.169256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.105143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.988498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.072832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.976872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.919602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.739175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.550177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.409630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.350549] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5041.356341] Lustre: Skipped 278 previous similar messages [ 5041.374877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5041.385544] Lustre: Skipped 278 previous similar messages [ 5041.393571] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5041.401245] Lustre: Skipped 278 previous similar messages [ 5041.407832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.415180] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5041.417551] Lustre: Skipped 278 previous similar messages [ 5042.278113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.263900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.212058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.043742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.863490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.685929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.439113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.178719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.972757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.724452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.534928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.321414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.108806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.856934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.647638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.459851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.356527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.152027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.946988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.834083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.668714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.448730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.195843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.074722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.884852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.664655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.560717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.373848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.254516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.037126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.913488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.799696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.746324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.436357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.212178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.083875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.911835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.679540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.527509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.288250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.033597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.726393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.418640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.295842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.122481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.033296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.769892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.496138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.208082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.011459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.817982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.628203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.351232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.219103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.046217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.910480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.616565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.461793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.399876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.227119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.002183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.974287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.831769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.787407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.632395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.436148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.232250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.938943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.770065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.651164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.404252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.149270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.914182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.724315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.449829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.318750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.179072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.031638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.712897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.477634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.211640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.983040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.701543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.365649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.041147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.828803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.683723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.416892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.209091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.016767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.759256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.531664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.399754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.288217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.978921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.815650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.523847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.307714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.026596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.791562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.495467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.205266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.927231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.627821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.320348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.280856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.172860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.978987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.831097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.622483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.388118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.235832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.119421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.009106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.012617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.829906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.569474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.405720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.403251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.286739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.989222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.796356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.535809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.385351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.282534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.320936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.201287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.124206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.075868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.165870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.065925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.849576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.643305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.428744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.322119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.234785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.939636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.706172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.578042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.666909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.360206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.376380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.116362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.946400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.701908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.404792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.253568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.124951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.020389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.866979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.729718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.569916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.491705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.279634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.034357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.773519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.707069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.420677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.324899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.329627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.170639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.027029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.963790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.795495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.620205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.336924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.296758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.234603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.120937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.937567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.804266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.715629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.608804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.341719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.218207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.064529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.069493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.884955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.826456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.716606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.715531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.621638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.542606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.266334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.051535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.897153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.763565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.540527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.400602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.347214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.249382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.043521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.818435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.633943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.390975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.137363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.821657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.667826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.435792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.291219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.060176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.858840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.598374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.371327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.103506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.842684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.564659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.326390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.061786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.811925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.540892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.205960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.945502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.714267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.461980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.299671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.127944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.908324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.717856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.479555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.286657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.218557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.006697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.913766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.786122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.546283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.319293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.209228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.058406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.901953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.531961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.396221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.176224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.129746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.068157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.946905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.712326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.622579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.403620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.382848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.357212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.249983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.029760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.836953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.689066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.599267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.348209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.253395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.050792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.006396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.882450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.721814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.538771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.411929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.228346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.002042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.726964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.586949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.332706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.276924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.153615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.009062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.783830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.616790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.328304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.000818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.859817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.772872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.691154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.558631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.336413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.185154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.210508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.138928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.998873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.942749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.757320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.656831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.524228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.426533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.378256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.555480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.392785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.388988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.219440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.042523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.867054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.910888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.817226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.848653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.691235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.631804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.468585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.773670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.133045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.001590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.776456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.713703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.676406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.576567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.420275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.521654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.332475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.279972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.308220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.074713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.872804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.680727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.471753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.166823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.890128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.629186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.544748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.454600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.332865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.212597] LustreError: 171848:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5305.228094] LustreError: 171848:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5305.233227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.087755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.044335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.877428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.926431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.844487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.856326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.642488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.462503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.351395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.411706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.152281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.052138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.064655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.053151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.756219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.593197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.350317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.120876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.877516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.744050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.555304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.441460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.348537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.218100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.999255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.898824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.736109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.768572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.614805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.444657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.320368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.151651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.872841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.622854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.462857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.382699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.215061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.114557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.868436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.722932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.508947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.318861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.055610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.045389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.827487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.694821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.400444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.242632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.127727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.116974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.139831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.183524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.093879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.170928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.898835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.726860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.790804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.975174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.106308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.865683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.770232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.641892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.712846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.683453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.587413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.359863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.160472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.060538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.875071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.714307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.687390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.595724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.379589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.159880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.073734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.834189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.700573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.676408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.664234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.503299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.449074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.306012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.400456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.458117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.263941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.176252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.974618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.984203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.886868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.785941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.903394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.696701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.772262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.104328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.063889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.116341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.994416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.941817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.744224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.610668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.488453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.330476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.142406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.899974] LustreError: 173651:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5762.909176] LustreError: 171848:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6433.254652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6433.261707] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6433.277334] Lustre: Skipped 419 previous similar messages [ 6433.291171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6434.794997] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6434.816248] LustreError: 225863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6434.840335] LustreError: 225863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6435.142351] Lustre: server umount lustre-OST0000 complete [ 6440.416740] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6440.434791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6445.772986] Lustre: server umount lustre-MDT0000 complete [ 6446.056341] LustreError: 172557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6446.075355] LustreError: 172557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6449.453887] LustreError: 187817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793152 with bad export cookie 17437680131341725628 [ 6449.464900] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6449.482530] LustreError: 187817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6449.965450] Lustre: server umount lustre-MDT0001 complete [ 6462.413621] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 6465.343547] Key type lgssc unregistered [ 6465.815707] LNet: 227077:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6465.828429] LNetError: 227077:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6465.858651] LNet: Removed LNI 192.168.206.144@tcp [ 6466.919290] Key type .llcrypt unregistered [ 6466.923726] Key type ._llcrypt unregistered [ 6493.445922] Key type ._llcrypt registered [ 6493.448616] Key type .llcrypt registered [ 6493.539538] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 6494.659464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6494.678706] alg: No test for adler32 (adler32-zlib) [ 6495.773574] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 6496.020552] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 6497.727160] Key type lgssc registered [ 6498.636185] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6505.200018] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6505.222682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6506.727472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6511.841242] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6516.218732] LustreError: 228494:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6516.244276] LustreError: 228494:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6521.245768] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6521.321896] LustreError: 228495:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6521.748939] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6526.291688] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6534.939889] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6542.096029] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6549.314610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6549.815495] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6551.866149] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6556.454985] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6564.011883] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6569.821875] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6571.927150] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6571.951421] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6571.959124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.978598] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6571.980928] LustreError: 228514:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6576.610381] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6576.613863] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6576.623617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6577.644459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6577.646948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6577.649900] Lustre: Skipped 1 previous similar message [ 6580.650674] Lustre: server umount lustre-OST0000 complete [ 6592.071093] LustreError: 228514:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6616.011536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6616.382096] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6617.387561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.416620] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6622.108673] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6627.687665] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.722417] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6630.201629] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6631.969472] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6638.025823] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d6459ade000.ost_server_uuid 50 [ 6639.430495] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8d6459ade000.ost_server_uuid in FULL state after 0 sec [ 6644.730322] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6644.741563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6644.758441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6648.295226] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6648.305956] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6648.307323] Lustre: Skipped 1 previous similar message [ 6649.608541] Lustre: server umount lustre-OST0000 complete [ 6654.944346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6654.959986] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6654.984258] Lustre: Skipped 1 previous similar message [ 6654.993255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6663.648787] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6663.654343] Lustre: Skipped 3 previous similar messages [ 6668.255590] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6668.489842] Lustre: server umount lustre-MDT0000 complete [ 6668.769123] LustreError: 228490:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6668.788258] LustreError: 228490:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6671.839812] LustreError: 228474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793374 with bad export cookie 3285787508210189427 [ 6671.847138] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6671.851390] LustreError: 228474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6672.091562] Lustre: server umount lustre-MDT0001 complete [ 6682.919668] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 6685.741237] Key type lgssc unregistered [ 6685.997826] LNet: 233708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6686.008505] LNetError: 233708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6686.029404] LNet: Removed LNI 192.168.206.144@tcp [ 6686.962215] Key type .llcrypt unregistered [ 6686.964643] Key type ._llcrypt unregistered [ 6719.931760] Key type ._llcrypt registered [ 6719.939476] Key type .llcrypt registered [ 6720.082867] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 6720.970972] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6720.991942] alg: No test for adler32 (adler32-zlib) [ 6722.075619] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 6722.275575] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 6723.943268] Key type lgssc registered [ 6725.028401] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6734.064895] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6734.076619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6735.513394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6739.915539] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6745.061432] LustreError: 235439:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6745.093749] LustreError: 235439:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6748.838840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6749.254402] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6753.706942] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6756.774905] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6763.125351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6763.502756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6769.864734] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6778.515905] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6778.632645] Lustre: lustre-OST0001: new disk, initializing [ 6778.635189] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6778.640822] Lustre: 237748:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6778.725918] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6780.803395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6784.700254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6784.714098] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6784.868175] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6786.343504] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6797.846478] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6806.118257] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6820.833172] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6820.839482] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6820.840380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6820.868052] Lustre: Skipped 3 previous similar messages [ 6825.775151] Lustre: server umount lustre-MDT0000 complete [ 6825.960089] LustreError: 235438:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6825.980078] LustreError: 235438:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6829.321340] LustreError: 235419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793532 with bad export cookie 14700412731627890706 [ 6829.325969] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6829.339831] LustreError: 235419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6829.648334] Lustre: server umount lustre-MDT0001 complete [ 6842.272312] Lustre: server umount lustre-OST0000 complete [ 6856.131097] Lustre: server umount lustre-OST0001 complete [ 6879.799650] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 6889.687452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6890.315307] LustreError: 240854:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6890.390091] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6895.498136] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6895.585599] LustreError: 240855:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6900.713197] LustreError: 240854:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6905.216494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6905.555509] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6909.881730] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6912.925889] Lustre: 241997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6919.522883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6927.257455] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6930.257159] LustreError: 242352:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6930.299366] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 6935.422435] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 6935.906355] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6936.196609] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6936.204474] Lustre: Skipped 1 previous similar message [ 6941.707290] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 6943.096469] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 6950.579509] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6954.346443] Lustre: 243866:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6964.336879] Lustre: server umount mylustre-MDT0000 complete [ 6967.270068] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6967.275779] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6967.279813] LustreError: 240851:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6967.279823] LustreError: 240851:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6967.291855] Lustre: Skipped 2 previous similar messages [ 6967.372210] LustreError: 240830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793670 with bad export cookie 14700412731627893555 [ 6967.376948] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6967.385714] LustreError: 240830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6972.387525] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6972.396577] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6972.414595] Lustre: Skipped 2 previous similar messages [ 6972.431096] Lustre: Skipped 4 previous similar messages [ 6973.763598] Lustre: server umount mylustre-MDT0001 complete [ 6976.915986] Lustre: server umount mylustre-OST0000 complete [ 6980.106423] Lustre: server umount mylustre-OST0001 complete [ 7001.415545] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7010.048060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7010.482166] LustreError: 246172:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7010.513689] LustreError: 246172:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7010.582282] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7014.445929] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7022.462422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7027.077715] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7029.919773] Lustre: 247315:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7036.502960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7043.477132] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7047.103489] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7052.990755] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7056.383173] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7056.415153] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7059.027382] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7066.617410] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7070.449536] Lustre: 249188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7079.392671] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7079.401801] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7079.424089] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7081.981291] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7081.988021] Lustre: Skipped 3 previous similar messages [ 7084.441395] Lustre: server umount tfs-MDT0000 complete [ 7087.083897] LustreError: 246172:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7087.119771] LustreError: 246172:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7088.284202] LustreError: 246148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793791 with bad export cookie 14700412731627895732 [ 7088.293915] LustreError: 246148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7088.297742] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7088.620645] Lustre: server umount tfs-MDT0001 complete [ 7102.848954] Lustre: server umount tfs-OST0000 complete [ 7116.822566] Lustre: server umount tfs-OST0001 complete [ 7137.501494] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7147.641277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7148.178648] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7148.183781] Lustre: Skipped 3 previous similar messages [ 7152.517087] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7161.607975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7166.002471] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7168.830788] Lustre: 252581:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7175.220366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7181.938500] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7187.008878] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 7189.591185] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7193.984015] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7193.984170] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7195.676353] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7203.392267] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7207.097163] Lustre: 254452:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7214.262160] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:25:15 (1787793915) [ 7234.016408] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7234.031499] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7234.047493] Lustre: Skipped 3 previous similar messages [ 7234.056402] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7236.770687] Lustre: server umount lustre-MDT0000 complete [ 7240.160408] LustreError: 251441:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7240.200285] LustreError: 251441:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7241.101087] LustreError: 251415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793944 with bad export cookie 14700412731627897650 [ 7241.115159] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7241.119981] LustreError: 251415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7245.300430] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7245.309157] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7245.323352] Lustre: Skipped 4 previous similar messages [ 7245.327292] Lustre: Skipped 4 previous similar messages [ 7247.687235] Lustre: server umount lustre-MDT0001 complete [ 7266.271112] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7266.395329] Lustre: server umount lustre-OST0000 complete [ 7276.492957] Lustre: server umount lustre-OST0001 complete [ 7285.137386] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 7296.355872] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7336.375653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7336.740154] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7336.787970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7336.841631] Lustre: lustre-MDT0000: new disk, initializing [ 7336.906898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7336.909990] Lustre: Skipped 3 previous similar messages [ 7336.938464] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7342.298535] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7353.946713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7354.047451] Lustre: 257638:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7354.157609] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7354.164075] Lustre: Skipped 1 previous similar message [ 7354.235543] Lustre: lustre-MDT0001: new disk, initializing [ 7354.359080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7354.373313] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7358.700716] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7369.766110] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7370.131423] Lustre: lustre-OST0000: new disk, initializing [ 7370.140766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7370.152479] Lustre: 259223:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7372.132582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7372.144877] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7372.293088] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7376.691959] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7386.971919] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7387.153238] Lustre: lustre-OST0001: new disk, initializing [ 7387.157580] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7387.163731] Lustre: 260095:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7389.186467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7389.200490] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7389.294515] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7393.642277] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7407.734919] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:28:28 (1787794108) [ 7424.143817] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:28:44 (1787794124) [ 7440.360866] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7440.391450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7440.396452] Lustre: Skipped 4 previous similar messages [ 7446.333976] Lustre: server umount lustre-OST0000 complete [ 7456.195394] Lustre: server umount lustre-MDT0000 complete [ 7459.801935] LustreError: 257631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794162 with bad export cookie 14700412731627899421 [ 7459.808754] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7459.818836] LustreError: 257631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7474.143134] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7474.419857] Lustre: server umount lustre-MDT0001 complete [ 7485.779253] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 7556.570332] Lustre: server umount lustre-OST0001 complete [ 7562.643656] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 7577.461510] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7609.549258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7609.722576] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7609.756350] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7609.845964] Lustre: lustre-MDT0000: new disk, initializing [ 7609.978207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7609.984250] Lustre: Skipped 3 previous similar messages [ 7609.996632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7614.069801] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7624.400388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7624.541656] Lustre: 264939:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7624.817606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7629.444679] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7639.030952] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7645.589614] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7652.336475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7652.620651] Lustre: lustre-OST0000: new disk, initializing [ 7652.623613] Lustre: Skipped 1 previous similar message [ 7652.628326] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7652.635281] Lustre: Skipped 2 previous similar messages [ 7652.639927] Lustre: 266907:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7654.242054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7654.248985] Lustre: Skipped 1 previous similar message [ 7654.256086] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7654.398222] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7659.132504] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7668.510676] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7674.858545] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7674.867882] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7674.888963] Lustre: Skipped 5 previous similar messages [ 7674.895091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7674.900568] Lustre: Skipped 11 previous similar messages [ 7681.055502] Lustre: server umount lustre-OST0000 complete [ 7691.241740] Lustre: server umount lustre-MDT0000 complete [ 7695.167789] LustreError: 265734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794398 with bad export cookie 14700412731627900898 [ 7695.179883] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7695.182232] LustreError: 265734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7695.558196] Lustre: server umount lustre-MDT0001 complete [ 7706.489978] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 7708.920488] Key type lgssc unregistered [ 7709.198844] LNet: 268673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7709.215227] LNetError: 268673:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7709.244241] LNet: Removed LNI 192.168.206.144@tcp [ 7710.080768] Key type .llcrypt unregistered [ 7710.087227] Key type ._llcrypt unregistered [ 7733.390197] Key type ._llcrypt registered [ 7733.394535] Key type .llcrypt registered [ 7733.480373] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7734.158900] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7734.230984] alg: No test for adler32 (adler32-zlib) [ 7735.258501] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 7735.472940] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 7737.191195] Key type lgssc registered [ 7738.494675] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7745.996844] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7746.025029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7747.414824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7751.732846] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7757.801201] LustreError: 270135:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7757.812816] LustreError: 270135:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7760.509211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7760.985461] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7765.467681] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7773.680890] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7780.031790] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7787.145722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7787.620947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7792.762172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7795.443969] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7803.238532] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7804.388698] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7808.482955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7808.492583] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7808.513215] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7811.555500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7811.558183] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7811.570031] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7813.449818] Lustre: server umount lustre-OST0000 complete [ 7817.184907] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7817.192335] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7817.208306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7817.212066] Lustre: Skipped 1 previous similar message [ 7821.793516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7821.794137] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7823.557328] Lustre: server umount lustre-MDT0000 complete [ 7826.917074] LustreError: 270135:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7826.954623] LustreError: 270135:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7827.039603] LustreError: 270116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794530 with bad export cookie 9352254165692897463 [ 7827.041282] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7827.061685] LustreError: 270116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7827.345585] Lustre: server umount lustre-MDT0001 complete [ 7838.610869] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 7842.382183] Key type lgssc unregistered [ 7842.701313] LNet: 273722:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7842.718038] LNetError: 273722:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7842.735831] LNet: Removed LNI 192.168.206.144@tcp [ 7843.902609] Key type .llcrypt unregistered [ 7843.905896] Key type ._llcrypt unregistered [ 7860.510580] Key type ._llcrypt registered [ 7860.513797] Key type .llcrypt registered [ 7860.623869] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7861.628415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7861.828978] alg: No test for adler32 (adler32-zlib) [ 7862.910923] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 7863.095116] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 7864.759562] Key type lgssc registered [ 7865.943302] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7874.201372] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7874.220576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7889.759638] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7894.879844] LustreError: 274918:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.144@tcp: failed processing log, type 4: rc = -110 [ 7921.503211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7926.511556] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 7933.445495] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7935.627504] Lustre: server umount lustre-OST0000 complete [ 7949.296464] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 7951.413360] Key type lgssc unregistered [ 7951.617317] LNet: 276552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7951.626946] LNetError: 276552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7951.645820] LNet: Removed LNI 192.168.206.144@tcp [ 7952.284939] Key type .llcrypt unregistered [ 7952.288757] Key type ._llcrypt unregistered [ 7986.311095] Key type ._llcrypt registered [ 7986.318715] Key type .llcrypt registered [ 7986.458201] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 7987.897728] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7987.920831] alg: No test for adler32 (adler32-zlib) [ 7989.043527] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 7989.297114] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 7990.943952] Key type lgssc registered [ 7991.815791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8004.102165] loop0: detected capacity change from 0 to 204800000 [ 8004.335399] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.342875] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8004.864600] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8006.037828] loop0: detected capacity change from 0 to 204800000 [ 8006.385412] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8006.415487] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8007.198936] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8008.415942] loop0: detected capacity change from 0 to 204800000 [ 8008.639075] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8008.660481] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8008.851230] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8009.865930] loop0: detected capacity change from 0 to 204800000 [ 8010.059219] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8010.084394] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8010.237303] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8011.357266] loop0: detected capacity change from 0 to 204800000 [ 8011.456984] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8015.319740] loop0: detected capacity change from 0 to 204800000 [ 8015.372475] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8019.759780] loop0: detected capacity change from 0 to 204800000 [ 8019.815128] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8023.475541] loop0: detected capacity change from 0 to 204800000 [ 8023.514793] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8027.150305] loop0: detected capacity change from 0 to 204800000 [ 8027.217160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8027.253976] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8027.339817] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8027.355217] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8027.365392] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8027.374711] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8027.387945] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8027.398819] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8027.407446] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8027.416048] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8027.440768] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8027.466430] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8027.486581] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8027.509167] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8027.521810] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8027.537071] Lustre: 279965:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8033.364766] Lustre: server umount MGS complete [ 8034.236158] loop0: detected capacity change from 0 to 204800000 [ 8034.362869] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8034.749208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8034.836047] Lustre: lustre-MDT0000: Aborting client recovery [ 8035.569301] loop1: detected capacity change from 0 to 204800000 [ 8035.752425] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8035.815207] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8035.826848] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8035.847485] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8035.858556] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8035.869806] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8035.879661] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8035.895181] Lustre: 280397:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8035.913714] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8036.103523] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8036.174751] Lustre: lustre-MDT0001: Aborting client recovery [ 8037.246873] loop2: detected capacity change from 0 to 204800000 [ 8037.389901] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8037.524323] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8037.536600] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8037.546784] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8037.556913] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8037.563562] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8037.572421] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8037.582619] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8037.688477] Lustre: Found index 0 for lustre-OST0000, updating log [ 8037.732828] Lustre: 280495:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8037.833090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8037.854982] Lustre: lustre-OST0000: Aborting recovery [ 8038.780062] loop3: detected capacity change from 0 to 204800000 [ 8038.952458] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8039.035436] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8039.043340] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8039.054922] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8039.063825] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8039.074381] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8039.081980] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8039.090550] Lustre: 280561:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8039.105575] Lustre: Found index 1 for lustre-OST0001, updating log [ 8039.126523] Lustre: 280566:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8039.190467] Lustre: lustre-OST0001: Aborting recovery [ 8044.323772] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8044.330915] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8044.339951] LustreError: 280341:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8044.357115] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8044.372587] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8046.386383] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8046.390528] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8047.702046] LustreError: 280308:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1165us: rc = -2 [ 8048.310500] LustreError: 280309:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 355us: rc = -2 [ 8048.346565] LustreError: 280309:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 8061.919786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8061.921575] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8061.924579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8061.952522] Lustre: Skipped 2 previous similar messages [ 8066.889942] Lustre: server umount lustre-MDT0000 complete [ 8067.041937] LustreError: 280311:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8067.054497] LustreError: 280311:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8067.547042] LustreError: 280290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794770 with bad export cookie 5950994601779703102 [ 8067.547468] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8067.558516] LustreError: 280290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8072.163648] LustreError: 280306:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8072.168286] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8072.173377] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8072.173386] Lustre: Skipped 3 previous similar messages [ 8072.186648] LustreError: 280306:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8072.219105] Lustre: Skipped 2 previous similar messages [ 8073.937725] Lustre: server umount lustre-MDT0001 complete [ 8089.056143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8089.143081] Lustre: server umount lustre-OST0000 complete [ 8096.215839] Lustre: server umount lustre-OST0001 complete [ 8103.998706] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:40:05 (1787794805) [ 8128.161823] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 8137.033547] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 8168.512583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8168.689468] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8168.718322] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8168.772263] Lustre: lustre-MDT0000: new disk, initializing [ 8168.831578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8168.836549] Lustre: Skipped 1 previous similar message [ 8168.851957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8172.682096] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8181.962347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8182.090440] Lustre: 284635:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8182.173380] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8182.178417] Lustre: Skipped 1 previous similar message [ 8182.273955] Lustre: lustre-MDT0001: new disk, initializing [ 8182.358647] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8182.395103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8182.401930] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8186.350917] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8194.229177] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8199.210522] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8205.141672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8205.419841] Lustre: lustre-OST0000: new disk, initializing [ 8205.433912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8205.444920] Lustre: 286600:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8205.530075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8211.276892] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8211.518763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8211.525689] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8211.589450] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8219.729703] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8224.595541] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8236.831206] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8241.868932] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8249.454488] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8288.739602] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8288.745519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8288.746942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8288.753535] Lustre: Skipped 1 previous similar message [ 8288.770559] Lustre: Skipped 2 previous similar messages [ 8293.763496] Lustre: server umount lustre-OST0000 complete [ 8298.984197] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8298.985564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8299.006312] Lustre: Skipped 1 previous similar message [ 8299.020600] Lustre: Skipped 1 previous similar message [ 8303.382362] Lustre: server umount lustre-MDT0000 complete [ 8304.097601] LustreError: 284640:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8304.112448] LustreError: 284640:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8306.920840] LustreError: 287457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795009 with bad export cookie 5950994601779705825 [ 8306.922813] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8306.933266] LustreError: 287457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8307.232841] Lustre: server umount lustre-MDT0001 complete [ 8314.888922] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8318.773506] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8323.685926] Lustre: server umount MGS complete [ 8334.874707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8335.425964] LustreError: 290514:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8335.448711] LustreError: 290527:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8335.542824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8340.319361] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8340.962080] LustreError: 290515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8349.090097] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8353.753629] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8359.849656] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8364.891100] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8370.115940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8370.364803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8370.374371] Lustre: Skipped 1 previous similar message [ 8375.379364] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8382.578696] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8410.611956] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8410.624354] LustreError: Skipped 1 previous similar message [ 8410.628653] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8410.655881] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8410.837983] Lustre: server umount lustre-OST0000 complete [ 8416.225368] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8421.857146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8421.867448] Lustre: Skipped 3 previous similar messages [ 8425.019453] Lustre: server umount lustre-MDT0000 complete [ 8426.978045] LustreError: 290514:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8426.987945] LustreError: 290514:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8428.714188] LustreError: 290494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795131 with bad export cookie 5950994601779707659 [ 8428.723466] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8428.726220] LustreError: 290494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8429.051619] Lustre: server umount lustre-MDT0001 complete [ 8439.427888] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 8442.187603] Key type lgssc unregistered [ 8442.497731] LNet: 294463:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8442.528214] LNetError: 294463:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8442.549925] LNet: Removed LNI 192.168.206.144@tcp [ 8443.420500] Key type .llcrypt unregistered [ 8443.422461] Key type ._llcrypt unregistered [ 8476.704874] Key type ._llcrypt registered [ 8476.707406] Key type .llcrypt registered [ 8476.816532] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 8490.830685] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 8491.984241] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8492.023033] alg: No test for adler32 (adler32-zlib) [ 8493.220380] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 8493.448962] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 8495.121654] Key type lgssc registered [ 8496.317815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8530.259970] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8530.287442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8531.658962] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8531.711876] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8531.839806] Lustre: lustre-MDT0000: new disk, initializing [ 8531.922388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8531.947222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8536.119958] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8546.404306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8546.567393] Lustre: 298421:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8546.637784] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8546.643026] Lustre: Skipped 1 previous similar message [ 8546.795934] Lustre: lustre-MDT0001: new disk, initializing [ 8546.923572] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8546.981812] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8547.001761] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8551.043929] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8559.876819] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8565.438519] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8571.622184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8571.960674] Lustre: lustre-OST0000: new disk, initializing [ 8571.963987] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8571.969256] Lustre: 300388:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8572.073768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8577.787398] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8578.180828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8578.205657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8578.377117] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8587.028628] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8594.520857] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8607.362191] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8613.030593] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8660.447964] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8660.449740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8660.454442] LustreError: Skipped 1 previous similar message [ 8660.461557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8660.469216] Lustre: Skipped 1 previous similar message [ 8665.424830] Lustre: server umount lustre-OST0000 complete [ 8669.664319] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8669.668652] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8669.675350] Lustre: Skipped 1 previous similar message [ 8669.680433] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8670.689744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8670.692544] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8674.582675] Lustre: server umount lustre-MDT0000 complete [ 8678.174284] LustreError: 301244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795381 with bad export cookie 6512545488809459065 [ 8678.190536] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8678.193993] LustreError: 301244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8678.740628] Lustre: server umount lustre-MDT0001 complete [ 8686.957505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8691.475853] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8696.022561] Lustre: server umount MGS complete [ 8705.279251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8705.786272] LustreError: 304302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8705.817387] LustreError: 304302:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8705.836805] LustreError: 304315:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8705.903297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8709.394213] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8711.137781] LustreError: 304303:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8715.235731] LustreError: 304302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8716.542679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8716.817679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8720.384537] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8726.186743] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8730.829069] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8737.454451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8737.827605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8744.059348] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8751.471952] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8777.696326] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8777.710647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8777.732651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8777.742891] Lustre: Skipped 1 previous similar message [ 8780.775350] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8783.021691] Lustre: server umount lustre-OST0000 complete [ 8788.450394] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8788.466080] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8788.486653] Lustre: Skipped 1 previous similar message [ 8788.499648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8788.512403] Lustre: Skipped 2 previous similar messages [ 8793.527539] Lustre: server umount lustre-MDT0000 complete [ 8796.138978] LustreError: 305030:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8796.157345] LustreError: 305030:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8797.347637] LustreError: 304284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795500 with bad export cookie 6512545488809460892 [ 8797.355181] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8797.358095] LustreError: 304284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8797.709442] Lustre: server umount lustre-MDT0001 complete [ 8807.996855] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [ 8810.729192] Key type lgssc unregistered [ 8811.113287] LNet: 308245:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8811.126257] LNetError: 308245:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8811.165991] LNet: Removed LNI 192.168.206.144@tcp [ 8811.989638] Key type .llcrypt unregistered [ 8811.996489] Key type ._llcrypt unregistered [ 8831.871897] Key type ._llcrypt registered [ 8831.873592] Key type .llcrypt registered [ 8832.007533] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 8847.934582] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 8849.478094] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8849.541513] alg: No test for adler32 (adler32-zlib) [ 8850.832543] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [ 8851.157658] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [ 8852.920137] Key type lgssc registered [ 8854.282082] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8880.580244] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 8896.292467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8896.346357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8897.518941] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8897.549822] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8897.620139] Lustre: lustre-MDT0000: new disk, initializing [ 8897.689476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8897.702849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8901.782406] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8911.560946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8911.796345] Lustre: lustre-OST0000: new disk, initializing [ 8911.800672] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8911.805462] Lustre: Skipped 1 previous similar message [ 8911.811184] Lustre: 312296:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8911.885155] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8917.667302] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 8919.137392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8919.145823] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8919.184289] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9758.695254] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9758.718986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9760.039428] Lustre: server umount lustre-MDT0000 complete [ 9763.902869] LustreError: 311484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796466 with bad export cookie 16101052849961115841 [ 9763.907074] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9763.918682] LustreError: 311484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9764.129522] Lustre: server umount lustre-OST0000 complete [ 9793.512304] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [ 9804.090152] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [ 9839.657812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9839.843791] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9839.902183] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9839.978744] Lustre: lustre-MDT0000: new disk, initializing [ 9840.045043] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9840.058550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9845.296669] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9856.482753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9856.611975] Lustre: 316648:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9856.695950] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9856.701566] Lustre: Skipped 1 previous similar message [ 9856.831848] Lustre: lustre-MDT0001: new disk, initializing [ 9856.952787] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9857.003325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9857.009448] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9861.881736] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9870.872358] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9876.872389] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9884.562727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9884.815596] Lustre: lustre-OST0000: new disk, initializing [ 9884.821609] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9884.829788] Lustre: 318614:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9884.925772] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9890.356332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9890.368233] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9890.464432] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9890.994108] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9900.201249] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9905.062444] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9905.319808] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9909.822819] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9910.010392] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9916.385033] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9916.400904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9916.405389] LustreError: Skipped 1 previous similar message [ 9916.412201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9916.423907] Lustre: Skipped 1 previous similar message [ 9916.452535] Lustre: Skipped 1 previous similar message [ 9918.215379] Lustre: server umount lustre-OST0000 complete [ 9923.561063] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9923.572182] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9923.583514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9926.624728] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9926.625760] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9928.811187] Lustre: server umount lustre-MDT0000 complete [ 9932.437885] LustreError: 316640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796635 with bad export cookie 16101052849964056660 [ 9932.444784] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9932.456129] LustreError: 316640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9932.840716] Lustre: server umount lustre-MDT0001 complete [ 9936.822580] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9938.467127] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 22:10:39 (1787796639) [ 9945.385213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9945.876793] LustreError: 320925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9945.901958] LustreError: 320925:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9945.977895] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9950.043446] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9951.211803] LustreError: 320926:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9955.297637] LustreError: 320925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9959.004837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9959.546512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9964.029767] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9971.845665] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9977.892414] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9984.612264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9984.985444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9991.298992] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [ 9999.071081] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10006.312718] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10006.521207] Lustre: lustre-OST0001: new disk, initializing [10006.533905] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10006.544499] Lustre: 323760:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10006.643393] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10012.717372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10012.729562] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10012.766502] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10013.144436] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10013.807767] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10013.838155] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10014.898244] LustreError: 324082:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10014.912143] LustreError: 324082:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10016.939591] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10016.950818] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10021.091136] LustreError: 324082:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10021.105810] LustreError: 324082:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10023.928761] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10024.131592] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10029.284804] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10029.292022] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10030.289947] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d64728f0000.ost_server_uuid 50 [10032.054142] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8d64728f0000.ost_server_uuid in FULL state after 0 sec [10037.701717] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8d64728f0000.ost_server_uuid 50 [10039.551983] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8d64728f0000.ost_server_uuid in FULL state after 0 sec [10041.680991] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10054.977777] Lustre: lustre-OST0001: Not available for connect from 192.168.206.44@tcp (stopping) [10054.984164] Lustre: Skipped 1 previous similar message [10057.201789] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10057.217616] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10057.236265] Lustre: Skipped 1 previous similar message [10060.919959] Lustre: server umount lustre-OST0001 complete [10064.381021] LustreError: 322745:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10064.399194] LustreError: 322745:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10069.473571] LustreError: 322748:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10069.476201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10069.490222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10069.490230] Lustre: Skipped 5 previous similar messages [10069.493906] LustreError: 322748:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10069.521177] Lustre: Skipped 1 previous similar message [10073.293412] Lustre: server umount lustre-OST0000 complete [10077.153636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10077.159757] LustreError: Skipped 2 previous similar messages [10083.187674] Lustre: server umount lustre-MDT0000 complete [10084.841694] LustreError: 321653:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10084.886470] LustreError: 321653:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10087.078261] LustreError: 320907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796790 with bad export cookie 16101052849964057542 [10087.080587] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10087.086770] LustreError: 320907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10087.425802] Lustre: server umount lustre-MDT0001 complete [10097.630684] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [10100.126743] Key type lgssc unregistered [10100.417754] LNet: 326450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10100.428811] LNetError: 326450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10100.467955] LNet: Removed LNI 192.168.206.144@tcp [10101.318785] Key type .llcrypt unregistered [10101.320707] Key type ._llcrypt unregistered [10128.806797] Key type ._llcrypt registered [10128.810582] Key type .llcrypt registered [10128.960530] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [10129.838341] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10130.039797] alg: No test for adler32 (adler32-zlib) [10131.093926] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [10131.404217] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [10133.055372] Key type lgssc registered [10134.233287] Lustre: Echo OBD driver; http://www.lustre.org/ [10141.223882] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10141.236757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10142.589107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10146.478855] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10152.931137] LustreError: 327871:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10152.944969] LustreError: 327871:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10153.540914] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10158.051656] LustreError: 327870:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10159.819575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10160.233502] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10164.153552] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10173.203964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10173.595596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10180.471763] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10185.925185] LustreError: 329553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10188.039494] LustreError: 329553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10188.052956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10188.069910] LustreError: 329553:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10188.598581] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10193.379536] LustreError: 329554:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10193.410493] LustreError: 329554:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10195.570425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10195.804097] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10201.105672] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10201.583497] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10209.381958] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10216.344374] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10216.610634] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10323.946435] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10323.954545] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10323.988169] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10323.994641] Lustre: Skipped 1 previous similar message [10326.616508] Lustre: server umount lustre-OST0001 complete [10329.067505] LustreError: 329555:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10329.089531] LustreError: 329555:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10334.186633] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10334.206020] Lustre: Skipped 1 previous similar message [10334.211129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10334.223384] Lustre: Skipped 1 previous similar message [10339.332368] Lustre: server umount lustre-OST0000 complete [10344.416573] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10344.424578] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10344.434910] LustreError: Skipped 1 previous similar message [10344.436741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10344.449575] Lustre: Skipped 2 previous similar messages [10344.474194] Lustre: Skipped 1 previous similar message [10349.537475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10349.544374] Lustre: Skipped 1 previous similar message [10349.841843] Lustre: server umount lustre-MDT0000 complete [10353.838823] LustreError: 331556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797056 with bad export cookie 12962636825762404878 [10353.841395] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10353.856410] LustreError: 331556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10354.241628] Lustre: server umount lustre-MDT0001 complete [10367.580254] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [10370.662944] Key type lgssc unregistered [10370.990715] LNet: 332819:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10370.998258] LNetError: 332819:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10372.070478] LNet: Removed LNI 192.168.206.144@tcp [10373.132722] Key type .llcrypt unregistered [10373.136801] Key type ._llcrypt unregistered [10420.908942] Key type ._llcrypt registered [10420.911794] Key type .llcrypt registered [10421.055219] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [10422.488464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10422.575649] alg: No test for adler32 (adler32-zlib) [10423.589402] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [10423.883206] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [10425.607588] Key type lgssc registered [10426.731950] Lustre: Echo OBD driver; http://www.lustre.org/ [10436.470541] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10436.499934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10437.980166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10442.002659] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10447.331561] LustreError: 335112:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10447.344034] LustreError: 335112:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10450.409633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10451.063709] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10455.958796] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10459.011634] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10465.940866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10466.341476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10473.101174] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10475.573364] LustreError: 337067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10480.621553] LustreError: 337067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10482.673400] LustreError: 337067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10482.703396] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10482.703852] LustreError: 337067:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10483.547808] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10483.840838] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10489.351904] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10489.912662] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10499.793986] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10508.947665] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10522.593872] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10522.612119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10522.641514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10525.157979] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10525.162231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10525.183657] Lustre: Skipped 2 previous similar messages [10525.204862] Lustre: Skipped 3 previous similar messages [10528.455243] Lustre: server umount lustre-MDT0000 complete [10530.279251] LustreError: 335851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10530.307244] LustreError: 335851:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10532.503203] LustreError: 335093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797235 with bad export cookie 10382089010520960127 [10532.526324] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10533.054288] Lustre: server umount lustre-MDT0001 complete [10547.710074] Lustre: server umount lustre-OST0000 complete [10562.037605] Lustre: server umount lustre-OST0001 complete [10580.059727] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [10591.810346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10592.332873] LustreError: 340380:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10592.448275] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10597.308518] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10607.061831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10607.614906] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10612.441437] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10615.700260] Lustre: 341521:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10623.311908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10623.859129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10630.488919] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10632.049150] LustreError: 341875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10632.069662] LustreError: 341875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10637.171151] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10639.138042] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10644.996754] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10647.367963] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10655.773930] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10659.699357] Lustre: 343391:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10665.447643] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10665.461730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10665.467573] Lustre: Skipped 3 previous similar messages [10665.472336] Lustre: Skipped 2 previous similar messages [10670.222758] Lustre: server umount lustre-MDT0000 complete [10670.562437] LustreError: 341112:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10670.582476] LustreError: 341112:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10673.843140] LustreError: 340361:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797376 with bad export cookie 10382089010520961828 [10673.850251] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10673.861486] LustreError: 340361:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10674.299453] Lustre: server umount lustre-MDT0001 complete [10689.111025] Lustre: server umount lustre-OST0000 complete [10691.872113] Lustre: 334375:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787797378/real 1787797378] req@ffff8e71434cc780 x1874641159310208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787797394 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10691.906172] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10693.242446] Lustre: server umount lustre-OST0001 complete [10697.279147] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10699.028637] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10701.397323] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:23:22 (1787797402) [10727.093633] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [10737.124441] loop0: detected capacity change from 0 to 1099511627776 [10737.839611] SGI XFS with ACLs, security attributes, quota, no debug enabled [10737.859677] *** VALIDATE xfs *** [10737.869464] XFS (loop0): Mounting V5 Filesystem [10737.903719] XFS (loop0): Ending clean mount [10739.520272] loop1: detected capacity change from 0 to 18691697672192 [10857.244936] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [10869.056876] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [10906.838827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10907.218728] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10907.268322] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10907.387356] Lustre: lustre-MDT0000: new disk, initializing [10907.538245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10907.547790] Lustre: Skipped 1 previous similar message [10907.580120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10911.924310] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10923.132853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10923.292135] Lustre: 348534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10923.373340] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10923.379415] Lustre: Skipped 1 previous similar message [10923.533823] Lustre: lustre-MDT0001: new disk, initializing [10923.744792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10923.754389] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10928.800444] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10939.061713] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10945.523573] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10952.913940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10953.279519] Lustre: lustre-OST0000: new disk, initializing [10953.285742] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10953.295687] Lustre: 350501:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10954.679441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10954.691021] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10954.885870] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10959.424941] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [10968.264459] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10973.236583] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10973.449418] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10977.755358] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10977.920504] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10980.324938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10980.325914] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10980.327919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10980.327925] Lustre: Skipped 1 previous similar message [10980.336508] Lustre: Skipped 1 previous similar message [10985.451158] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10985.461621] Lustre: Skipped 2 previous similar messages [10985.927067] Lustre: server umount lustre-OST0000 complete [10990.561800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10990.562433] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10990.593334] Lustre: Skipped 1 previous similar message [10995.690274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10995.703452] Lustre: Skipped 3 previous similar messages [10996.175249] Lustre: server umount lustre-MDT0000 complete [11000.155131] LustreError: 348527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797703 with bad export cookie 10382089010520963382 [11000.156351] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11000.170044] LustreError: 348527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11000.639091] Lustre: server umount lustre-MDT0001 complete [11009.243902] XFS (loop0): Unmounting Filesystem [11014.276640] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:28:35 (1787797715) [11022.124916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11022.633350] LustreError: 353060:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11022.755972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11022.760415] Lustre: Skipped 2 previous similar messages [11027.855824] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11036.684769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11042.100654] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11050.511520] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11056.191447] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11062.801247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11069.644485] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11076.777885] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11091.937862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11091.962294] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11091.970159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11091.984181] Lustre: Skipped 1 previous similar message [11092.010393] Lustre: Skipped 1 previous similar message [11094.785699] Lustre: server umount lustre-OST0000 complete [11098.596318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11104.845847] Lustre: server umount lustre-MDT0000 complete [11108.225806] LustreError: 353041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797811 with bad export cookie 10382089010520964264 [11108.227348] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11108.235118] LustreError: 353041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11108.676582] Lustre: server umount lustre-MDT0001 complete [11119.842759] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [11122.826729] Key type lgssc unregistered [11123.070348] LNet: 356693:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11123.076354] LNetError: 356693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11123.095972] LNet: Removed LNI 192.168.206.144@tcp [11123.779147] Key type .llcrypt unregistered [11123.782507] Key type ._llcrypt unregistered [11151.624776] Key type ._llcrypt registered [11151.626701] Key type .llcrypt registered [11151.715223] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [11152.882295] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11152.910839] alg: No test for adler32 (adler32-zlib) [11153.929814] Lustre: Lustre: Build Version: 2.17.57_52_gc224de1 [11154.157977] LNet: Added LNI 192.168.206.144@tcp [8/256/0/180] [11155.839257] Key type lgssc registered [11157.690240] Lustre: Echo OBD driver; http://www.lustre.org/ [11165.905453] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11165.956496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11167.467297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11172.889854] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11177.955144] LustreError: 358263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11177.975126] LustreError: 358263:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11182.292683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11182.629967] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11186.925601] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11194.663539] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11200.108687] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11206.894889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11207.143399] LustreError: 360084:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11207.286697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11213.956513] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11221.809819] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11233.253483] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11233.280526] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11233.292258] Lustre: Skipped 1 previous similar message [11234.630808] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.44@tcp (stopping) [11236.122447] Lustre: server umount lustre-MDT0001 complete [11238.371866] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11238.386649] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11243.494769] LustreError: 358263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11243.556292] LustreError: 358263:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11248.173034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11248.262362] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11248.267680] Lustre: Found index 1 for lustre-MDT0001, updating log [11248.340455] Lustre: 358251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11248.747869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11253.765148] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11254.061368] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11255.802667] LustreError: 361797:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11255.820740] LustreError: 361797:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11255.839593] LustreError: 358255:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11255.931720] LustreError: 361812:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.144@tcp: cfg command failed: rc = -17 [11255.941185] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11255.941185] [11263.972592] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11263.972897] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11263.983958] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11263.995780] Lustre: Skipped 2 previous similar messages [11269.090664] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11269.103225] Lustre: Skipped 3 previous similar messages [11274.208930] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11274.217389] Lustre: Skipped 2 previous similar messages [11278.304699] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11278.597186] Lustre: server umount lustre-MDT0001 complete [11279.335512] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11279.361809] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11289.577848] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11289.612844] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11290.509411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11290.691978] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11290.699964] Lustre: Found index 1 for lustre-MDT0001, updating log [11290.768400] Lustre: 358250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11290.931557] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.44@tcp (not set up) [11290.939329] Lustre: Skipped 1 previous similar message [11291.144219] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11296.250540] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11296.638187] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11298.813728] LustreError: 362898:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11298.826826] LustreError: 362898:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11298.838340] LustreError: 358255:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11298.852093] LustreError: 358255:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11298.952965] LustreError: 362904:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11298.967879] LustreError: 362904:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11298.981524] LustreError: 362904:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.144@tcp: cfg command failed: rc = -17 [11298.988950] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.144@tcp [11298.988950] [11603.267446] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.44@tcp (stopping) [11603.429781] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11603.430570] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11617.247947] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11617.469495] Lustre: server umount lustre-MDT0001 complete [11618.623388] LustreError: 364564:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11628.152428] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11628.250058] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11628.254145] Lustre: Found index 1 for lustre-MDT0001, updating log [11628.286790] Lustre: 358251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11628.605861] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11633.026950] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [11633.660226] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11637.754914] LustreError: 371513:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11637.768909] LustreError: 371513:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11637.787201] LustreError: 358255:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11637.796800] LustreError: 358255:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11637.858020] LustreError: 371516:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11637.884030] LustreError: 371516:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.144@tcp: cfg command failed: rc = -114 [11637.899849] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11637.899849] [11956.195931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11956.200755] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11956.203310] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11956.203320] Lustre: Skipped 8 previous similar messages [11956.241719] Lustre: Skipped 3 previous similar messages [11962.244340] Lustre: server umount lustre-MDT0000 complete [11966.439323] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11966.485864] LustreError: 358262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [11966.934397] LustreError: 361449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798669 with bad export cookie 4302290892588786100 [11966.952772] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11966.982722] LustreError: 361449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11971.553048] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11973.662512] Lustre: server umount lustre-MDT0001 complete [11977.943564] Lustre: server umount lustre-OST0000 complete [11989.251836] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [12001.100958] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [12039.558451] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:45:40 (1787798740) [12046.074368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12046.279565] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12046.311869] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12046.386150] Lustre: lustre-MDT0000: new disk, initializing [12046.488962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12046.506892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12050.787863] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [12061.103050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12061.252317] Lustre: 382096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12061.318811] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12061.333647] Lustre: Skipped 1 previous similar message [12061.434678] Lustre: lustre-MDT0001: new disk, initializing [12061.612996] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12061.662249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12061.681962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12066.195206] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [12076.538093] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12083.303591] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12090.263688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12090.524505] Lustre: lustre-OST0000: new disk, initializing [12090.531410] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12090.540861] Lustre: 384066:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12090.609685] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12098.201681] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_default_debug -1 all [12100.174420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12100.189946] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12100.343469] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12108.808994] Lustre: DEBUG MARKER: oleg644-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12117.987117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12117.988582] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12117.993699] LustreError: Skipped 1 previous similar message [12118.010581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12118.014711] Lustre: Skipped 6 previous similar messages [12119.233824] Lustre: server umount lustre-MDT0000 complete [12121.058754] LustreError: 382109:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12121.072918] LustreError: 382109:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12122.824735] LustreError: 384076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798825 with bad export cookie 4302290892588788053 [12122.829436] LustreError: MGC192.168.206.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12122.845611] LustreError: 384076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12122.871690] LustreError: 382904:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12123.263570] Lustre: server umount lustre-MDT0001 complete [12137.392381] Lustre: server umount lustre-OST0000 complete [12166.620498] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing set_hostid [12177.457563] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing load_modules_local [12209.350425] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11921 sec ========= 22:48:30 (1787798910) [12211.357768] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:48:32 (1787798912) === [12215.025616] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:48:35 (1787798915) === [12254.783305] Lustre: DEBUG MARKER: oleg644-server.virtnet: executing unload_modules_local [12257.475849] Key type lgssc unregistered [12257.794893] LNet: 390435:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12257.801809] LNetError: 390435:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12257.817539] LNet: Removed LNI 192.168.206.144@tcp [12258.505606] Key type .llcrypt unregistered [12258.507877] Key type ._llcrypt unregistered