[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 443877703 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003181] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005011] kvm-guest: setup PV IPIs [ 0.007719] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008015] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009006] pid_max: default: 32768 minimum: 301 [ 0.011068] LSM: Security Framework initializing [ 0.011694] Yama: becoming mindful. [ 0.012019] SELinux: Initializing. [ 0.012965] *** VALIDATE selinux *** [ 0.019618] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.023295] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024131] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025074] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026071] *** VALIDATE tmpfs *** [ 0.027221] *** VALIDATE proc *** [ 0.028168] *** VALIDATE cgroup *** [ 0.029005] *** VALIDATE cgroup2 *** [ 0.030225] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.031139] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.032005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.033021] Spectre V2 : User space: Vulnerable [ 0.033912] Speculative Store Bypass: Vulnerable [ 0.035180] debug: unmapping init [mem 0xffffffffa2459000-0xffffffffa2460fff] [ 0.037849] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.038703] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.039019] ... version: 2 [ 0.040009] ... bit width: 48 [ 0.041008] ... generic registers: 4 [ 0.042008] ... value mask: 0000ffffffffffff [ 0.043008] ... max period: 00007fffffffffff [ 0.044008] ... fixed-purpose events: 3 [ 0.045007] ... event mask: 000000070000000f [ 0.046291] rcu: Hierarchical SRCU implementation. [ 0.048442] smp: Bringing up secondary CPUs ... [ 0.049529] x86: Booting SMP configuration: [ 0.050014] .... node #0, CPUs: #1 #2 #3 [ 0.061375] smp: Brought up 1 node, 4 CPUs [ 0.063010] smpboot: Max logical packages: 1 [ 0.064014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.142375] node 0 deferred pages initialised in 76ms [ 0.151033] devtmpfs: initialized [ 0.152216] x86/mm: Memory block size: 128MB [ 0.154898] gcov: version magic: 0x41383552 [ 0.157358] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.158092] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.160249] pinctrl core: initialized pinctrl subsystem [ 0.162164] [ 0.162624] ************************************************************* [ 0.164010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166009] ** ** [ 0.168011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.170008] ** ** [ 0.172011] ** This means that this kernel is built to expose internal ** [ 0.174009] ** IOMMU data structures, which may compromise security on ** [ 0.176010] ** your system. ** [ 0.178011] ** ** [ 0.181014] ** If you see this message and you are not debugging the ** [ 0.183011] ** kernel, report this immediately to your vendor! ** [ 0.185014] ** ** [ 0.188014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.190010] ************************************************************* [ 0.192649] NET: Registered protocol family 16 [ 0.194451] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.196047] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.198055] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.202054] cpuidle: using governor menu [ 0.204000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.208248] PCI: Using configuration type 1 for base access [ 0.212144] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.230151] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.234089] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.239163] cryptd: max_cpu_qlen set to 1000 [ 0.241352] ACPI: Added _OSI(Module Device) [ 0.243030] ACPI: Added _OSI(Processor Device) [ 0.245015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.247023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.252331] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.258364] ACPI: Interpreter enabled [ 0.259058] ACPI: PM: (supports S0 S3 S4 S5) [ 0.261013] ACPI: Using IOAPIC for interrupt routing [ 0.263093] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.266509] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.282937] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.285030] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.287016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.290077] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.294335] acpiphp: Slot [2] registered [ 0.295107] acpiphp: Slot [5] registered [ 0.297052] acpiphp: Slot [6] registered [ 0.298021] acpiphp: Slot [7] registered [ 0.299115] acpiphp: Slot [8] registered [ 0.300085] acpiphp: Slot [9] registered [ 0.301042] acpiphp: Slot [10] registered [ 0.302070] acpiphp: Slot [3] registered [ 0.303063] acpiphp: Slot [4] registered [ 0.305107] acpiphp: Slot [11] registered [ 0.307104] acpiphp: Slot [12] registered [ 0.309096] acpiphp: Slot [13] registered [ 0.310103] acpiphp: Slot [14] registered [ 0.312084] acpiphp: Slot [15] registered [ 0.313083] acpiphp: Slot [16] registered [ 0.315076] acpiphp: Slot [17] registered [ 0.317118] acpiphp: Slot [18] registered [ 0.319115] acpiphp: Slot [19] registered [ 0.320110] acpiphp: Slot [20] registered [ 0.321000] acpiphp: Slot [21] registered [ 0.321000] acpiphp: Slot [22] registered [ 0.323087] acpiphp: Slot [23] registered [ 0.324076] acpiphp: Slot [24] registered [ 0.326085] acpiphp: Slot [25] registered [ 0.327083] acpiphp: Slot [26] registered [ 0.329145] acpiphp: Slot [27] registered [ 0.330099] acpiphp: Slot [28] registered [ 0.332092] acpiphp: Slot [29] registered [ 0.333077] acpiphp: Slot [30] registered [ 0.335087] acpiphp: Slot [31] registered [ 0.336062] PCI host bridge to bus 0000:00 [ 0.338015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.340019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.342056] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.345022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.348025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.351028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.353174] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.358370] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.365156] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.373622] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.378408] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.380013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.382019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.384014] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.385556] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.390253] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.393036] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.396046] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.401017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.413017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.416020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.422865] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.437016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.450025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.483017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.527763] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.534012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.539012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.550017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.559000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.565015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.572021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.588018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.596026] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.600021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.612028] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.634022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.645251] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.654016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.667023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.699017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.711661] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.719016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.727015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.743014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.765256] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.768332] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.772407] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.774285] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.776191] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.780246] iommu: Default domain type: Passthrough [ 0.781435] SCSI subsystem initialized [ 0.783133] ACPI: bus type USB registered [ 0.785096] usbcore: registered new interface driver usbfs [ 0.787110] usbcore: registered new interface driver hub [ 0.789111] usbcore: registered new device driver usb [ 0.790221] pps_core: LinuxPPS API ver. 1 registered [ 0.792009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.794055] PTP clock support registered [ 0.796519] EDAC MC: Ver: 3.0.0 [ 0.798205] PCI: Using ACPI for IRQ routing [ 0.800985] NetLabel: Initializing [ 0.802068] NetLabel: domain hash size = 128 [ 0.804009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.806127] NetLabel: unlabeled traffic allowed by default [ 0.808269] vgaarb: loaded [ 0.810290] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.812013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.822319] clocksource: Switched to clocksource kvm-clock [ 0.940457] VFS: Disk quotas dquot_6.6.0 [ 0.942246] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.944524] *** VALIDATE ramfs *** [ 0.945767] *** VALIDATE hugetlbfs *** [ 0.947416] pnp: PnP ACPI init [ 0.950100] pnp: PnP ACPI: found 6 devices [ 0.967192] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.970773] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.973162] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.975512] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.978183] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.980801] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.983924] NET: Registered protocol family 2 [ 0.986535] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.991828] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.995776] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.001329] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.004837] TCP: Hash tables configured (established 65536 bind 65536) [ 1.007910] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.011029] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.014324] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.017812] NET: Registered protocol family 1 [ 1.021253] RPC: Registered named UNIX socket transport module. [ 1.023990] RPC: Registered udp transport module. [ 1.025917] RPC: Registered tcp transport module. [ 1.027879] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.030738] NET: Registered protocol family 44 [ 1.034165] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.036449] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.038311] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.040922] PCI: CLS 0 bytes, default 64 [ 1.042876] Unpacking initramfs... [ 2.544383] debug: unmapping init [mem 0xffff8f663cc54000-0xffff8f663ffbffff] [ 2.549292] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.551386] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.554194] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.048262] Initialise system trusted keyrings [ 3.049484] Key type blacklist registered [ 3.050922] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.060025] zbud: loaded [ 3.062854] *** VALIDATE nfs *** [ 3.064314] *** VALIDATE nfs4 *** [ 3.066169] pstore: using deflate compression [ 3.069260] Platform Keyring initialized [ 3.166399] NET: Registered protocol family 38 [ 3.168656] Key type asymmetric registered [ 3.176956] Asymmetric key parser 'x509' registered [ 3.179103] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.184312] io scheduler mq-deadline registered [ 3.185535] io scheduler kyber registered [ 3.186661] io scheduler bfq registered [ 3.188925] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.192214] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.195527] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.198385] ACPI: Power Button [PWRF] [ 3.204089] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.211449] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.224466] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.231771] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.246730] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.276745] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.306661] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.311188] Non-volatile memory driver v1.3 [ 3.312795] Linux agpgart interface v0.103 [ 3.342832] virtio_blk virtio1: [vda] 136000 512-byte logical blocks (69.6 MB/66.4 MiB) [ 3.345644] vda: detected capacity change from 0 to 69632000 [ 3.366154] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.370305] vdb: detected capacity change from 0 to 1073741824 [ 3.389653] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.392495] vdc: detected capacity change from 0 to 2621440000 [ 3.410178] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.413325] vdd: detected capacity change from 0 to 2621440000 [ 3.562359] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.599952] vde: detected capacity change from 0 to 4294967296 [ 3.668858] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.672042] vdf: detected capacity change from 0 to 4294967296 [ 3.681842] libphy: Fixed MDIO Bus: probed [ 3.687881] usbcore: registered new interface driver usbserial_generic [ 3.691100] usbserial: USB Serial support registered for generic [ 3.693655] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.698312] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.700925] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.703673] mousedev: PS/2 mouse device common for all mice [ 3.707229] rtc_cmos 00:05: RTC can wake from S4 [ 3.711699] rtc_cmos 00:05: registered as rtc0 [ 3.714115] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.718923] intel_pstate: CPU model not supported [ 3.720196] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.727411] hid: raw HID events driver (C) Jiri Kosina [ 3.730405] usbcore: registered new interface driver usbhid [ 3.733365] usbhid: USB HID core driver [ 3.735750] drop_monitor: Initializing network drop monitor service [ 3.736070] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.740691] Initializing XFRM netlink socket [ 3.749546] NET: Registered protocol family 10 [ 3.753529] Segment Routing with IPv6 [ 3.754963] NET: Registered protocol family 17 [ 3.758866] mpls_gso: MPLS GSO support [ 3.763351] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.766696] RAS: Correctable Errors collector initialized. [ 3.771764] AVX version of gcm_enc/dec engaged. [ 3.773132] AES CTR mode by8 optimization enabled [ 3.856961] sched_clock: Marking stable (3856883719, 0)->(4667309364, -810425645) [ 3.861459] registered taskstats version 1 [ 3.863722] Loading compiled-in X.509 certificates [ 3.866283] zswap: loaded using pool lzo/zbud [ 3.892628] Key type big_key registered [ 3.906853] Key type encrypted registered [ 3.908830] ima: No TPM chip found, activating TPM-bypass! [ 3.911262] ima: Allocated hash algorithm: sha1 [ 3.913389] ima: No architecture policies found [ 3.915414] evm: Initialising EVM extended attributes: [ 3.917696] evm: security.selinux [ 3.919202] evm: security.ima [ 3.920421] evm: security.capability [ 3.921983] evm: HMAC attrs: 0x1 [ 3.924639] rtc_cmos 00:05: setting system clock to 2026-05-29 20:18:30 UTC (1780085910) [ 3.931942] debug: unmapping init [mem 0xffffffffa3403000-0xffffffffa35fffff] [ 3.934877] debug: unmapping init [mem 0xffffffffa2182000-0xffffffffa2458fff] [ 3.944496] Write protecting the kernel read-only data: 28672k [ 3.969678] debug: unmapping init [mem 0xffffffffa0803000-0xffffffffa09fffff] [ 3.976654] debug: unmapping init [mem 0xffffffffa1114000-0xffffffffa11fffff] [ 4.040310] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.057893] systemd[1]: Detected virtualization kvm. [ 4.060326] systemd[1]: Detected architecture x86-64. [ 4.062605] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.093079] systemd[1]: No hostname configured. [ 4.094771] systemd[1]: Set hostname to . [ 4.097130] random: systemd: uninitialized urandom read (16 bytes read) [ 4.100114] systemd[1]: Initializing machine ID from random generator. [ 4.209135] random: ln: uninitialized urandom read (6 bytes read) [ 4.340771] random: systemd: uninitialized urandom read (16 bytes read) [ 4.343730] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.353710] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.361760] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.534958] device-mapper: uevent: version 1.0.3 [ 5.537475] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.456434] virtio_net virtio0 ens2: renamed from eth0 [ 6.509273] random: fast init done [ 6.660693] scsi host0: ata_piix [ 6.704741] scsi host1: ata_piix [ 6.707973] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.711665] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.602152] random: crng init done [ 11.604572] random: 7 urandom warning(s) missed due to ratelimiting [ 13.163544] dracut-initqueue[581]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 15.269274] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 16.976914] printk: systemd: 26 output lines suppressed due to ratelimiting [ 17.327277] SELinux: Disabled at runtime. [ 17.388994] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 17.398122] systemd[1]: Detected virtualization kvm. [ 17.400121] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 18.209515] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.213755] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.221799] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.227027] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.230612] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.243622] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.254624] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Process Core Dump Socket. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ 18.571239] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ OK ] Reached target Paths. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 19.653553] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.521559] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 20.556971] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.182584] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.362742] EDAC sbridge: Ver: 1.1.2 [ 24.309685] Key type dns_resolver registered [ 24.782349] NFS: Registering the id_resolver key type [ 24.783901] Key type id_resolver registered [ 24.784959] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg160-server login: [ 72.963179] libcfs: loading out-of-tree module taints kernel. [ 72.992739] Key type ._llcrypt registered [ 72.993961] Key type .llcrypt registered [ 73.049842] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 82.938428] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 83.853763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 83.859323] alg: No test for adler32 (adler32-zlib) [ 84.946289] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 85.411548] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 87.071663] Key type lgssc registered [ 87.908175] Lustre: Echo OBD driver; http://www.lustre.org/ [ 97.066563] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 116.440536] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 123.195487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 123.211185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 124.369240] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 124.391563] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 124.453789] Lustre: lustre-MDT0000: new disk, initializing [ 124.519070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 124.532204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 126.612731] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 134.272586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 134.333200] Lustre: 6479:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 134.357942] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 134.361216] Lustre: Skipped 1 previous similar message [ 134.404477] Lustre: lustre-MDT0001: new disk, initializing [ 134.437786] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 134.454949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 134.460302] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 136.509787] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 139.670383] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 145.024543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 145.148896] Lustre: lustre-OST0000: new disk, initializing [ 145.151330] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 145.156026] Lustre: 8384:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 145.186803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 148.037618] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 148.042469] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 148.095737] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 148.545594] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 156.921094] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 156.978188] Lustre: lustre-OST0001: new disk, initializing [ 156.980988] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 156.984626] Lustre: 9438:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 157.019494] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 160.124586] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 166.920448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 166.930330] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 166.979552] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 168.283726] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 174.333838] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 177.745392] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing check_logdir /tmp/testlogs/ [ 180.009588] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing yml_node [ 182.038475] Lustre: DEBUG MARKER: Client: 2.17.53.53 [ 183.438463] Lustre: DEBUG MARKER: MDS: 2.17.53.53 [ 184.934814] Lustre: DEBUG MARKER: OSS: 2.17.53.53 [ 186.003871] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri May 29 16:21:31 EDT 2026 [ 196.066247] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 196.986550] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 211.425930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 211.430361] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 211.436189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 212.962433] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 212.962682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 212.970682] Lustre: Skipped 1 previous similar message [ 212.980844] Lustre: Skipped 3 previous similar messages [ 215.950782] Lustre: server umount lustre-MDT0000 complete [ 217.927698] LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086124 with bad export cookie 9866462866709438162 [ 217.929141] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 217.933583] LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 218.083670] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 218.083980] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 218.094088] Lustre: Skipped 2 previous similar messages [ 218.101223] Lustre: Skipped 1 previous similar message [ 222.111838] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 222.119277] Lustre: Skipped 1 previous similar message [ 224.172960] Lustre: server umount lustre-MDT0001 complete [ 226.400808] Lustre: server umount lustre-OST0000 complete [ 228.644381] Lustre: server umount lustre-OST0001 complete [ 232.752707] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 237.913505] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 259.143570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 259.310464] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 259.334724] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 259.386327] Lustre: lustre-MDT0000: new disk, initializing [ 259.438515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 259.451258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 262.431100] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 269.389587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 269.451052] Lustre: 14945:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 269.488272] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 269.491516] Lustre: Skipped 1 previous similar message [ 269.534536] Lustre: lustre-MDT0001: new disk, initializing [ 269.597925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 269.623023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 269.630257] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 271.743682] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 276.223969] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 278.328557] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 282.361760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 282.554447] Lustre: 16685:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 284.442804] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 284.528500] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 285.503048] hrtimer: interrupt took 10036448 ns [ 286.202505] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 291.348309] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 293.907895] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 294.041117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 296.593459] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 296.735523] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 300.001893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 300.008225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 300.010381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 300.025905] Lustre: Skipped 1 previous similar message [ 300.037290] Lustre: Skipped 1 previous similar message [ 304.236882] Lustre: server umount lustre-OST0000 complete [ 310.242425] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 310.244685] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 310.255472] Lustre: Skipped 1 previous similar message [ 312.612123] Lustre: server umount lustre-MDT0000 complete [ 314.727259] LustreError: 16684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086221 with bad export cookie 9866462866709439695 [ 314.731135] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 314.733789] LustreError: 16684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 314.980053] Lustre: server umount lustre-MDT0001 complete [ 317.788167] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:23:43 (1780086223) [ 321.580713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 321.838052] LustreError: 18801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 321.852427] LustreError: 18801:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 321.897440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 321.904660] Lustre: Skipped 1 previous similar message [ 324.247760] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 327.137452] LustreError: 18802:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 329.454908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 332.315775] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 335.435839] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 337.618166] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 341.766413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 345.024317] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 347.896098] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 359.623436] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 365.543245] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 365.546940] LustreError: Skipped 1 previous similar message [ 365.548153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.549560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 365.552848] Lustre: Skipped 1 previous similar message [ 365.555314] Lustre: Skipped 1 previous similar message [ 365.816289] Lustre: server umount lustre-MDT0000 complete [ 368.209268] LustreError: 18781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086274 with bad export cookie 9866462866709440577 [ 368.217747] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 368.218620] LustreError: 18781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 368.428398] Lustre: server umount lustre-MDT0001 complete [ 381.365862] Lustre: server umount lustre-OST0000 complete [ 394.658632] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 400.742926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 400.956749] LustreError: 22845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 401.001943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 401.006863] Lustre: Skipped 2 previous similar messages [ 403.310570] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 406.499258] LustreError: 22846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 409.134031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 412.129829] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 414.186459] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 418.760542] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 422.661225] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 427.788364] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 427.890727] Lustre: lustre-OST0001: new disk, initializing [ 427.893394] Lustre: Skipped 1 previous similar message [ 427.896924] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 427.899881] Lustre: Skipped 1 previous similar message [ 427.903782] Lustre: 25106:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 429.271627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 429.280898] Lustre: Skipped 1 previous similar message [ 429.285684] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 429.367546] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 431.772937] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 438.859984] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 441.118614] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 449.000830] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 449.006574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 449.008632] Lustre: Skipped 3 previous similar messages [ 449.022747] Lustre: Skipped 4 previous similar messages [ 453.758441] Lustre: server umount lustre-MDT0000 complete [ 454.121223] LustreError: 22841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 454.135172] LustreError: 22841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 456.203055] LustreError: 22825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086362 with bad export cookie 9866462866709441424 [ 456.206086] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 456.211904] LustreError: 22825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 456.418581] Lustre: server umount lustre-MDT0001 complete [ 468.546070] Lustre: server umount lustre-OST0000 complete [ 481.226089] Lustre: server umount lustre-OST0001 complete [ 490.097688] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 495.996861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 496.305500] LustreError: 28037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 496.370821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 496.374370] Lustre: Skipped 3 previous similar messages [ 498.865508] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 503.624517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 506.035888] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 507.628458] Lustre: 29151:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 510.912939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 514.107499] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 514.220729] LustreError: 29505:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.233560] LustreError: 29505:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 518.759440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 522.083195] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 526.418323] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 528.378402] Lustre: 30995:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 531.426626] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 531.429299] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 531.437474] Lustre: Skipped 4 previous similar messages [ 531.444729] Lustre: Skipped 3 previous similar messages [ 537.065404] Lustre: server umount lustre-MDT0000 complete [ 539.039776] LustreError: 28769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086445 with bad export cookie 9866462866709442712 [ 539.041831] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 539.046454] LustreError: 28769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 539.216520] Lustre: server umount lustre-MDT0001 complete [ 550.918781] Lustre: server umount lustre-OST0000 complete [ 563.058835] Lustre: server umount lustre-OST0001 complete [ 567.195177] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:27:53 (1780086473) [ 579.788949] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 584.525270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 584.730523] LustreError: 33774:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.612744] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 590.402898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 592.192527] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 593.380484] Lustre: 34889:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 596.465078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 599.481377] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 603.389881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 605.780516] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 615.099464] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 617.300449] Lustre: 36732:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 620.002681] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 620.012557] Lustre: Skipped 2 previous similar messages [ 626.180185] Lustre: server umount lustre-MDT0000 complete [ 627.974259] LustreError: 34508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086534 with bad export cookie 9866462866709443818 [ 627.976333] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 627.980451] LustreError: 34508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 628.124683] Lustre: server umount lustre-MDT0001 complete [ 640.457599] Lustre: server umount lustre-OST0000 complete [ 652.237738] Lustre: server umount lustre-OST0001 complete [ 655.774790] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:29:21 (1780086561) [ 661.912258] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 666.383301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 666.555984] LustreError: 38857:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 666.565056] LustreError: 38857:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 666.597626] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 666.600018] Lustre: Skipped 7 previous similar messages [ 668.250278] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 671.863343] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 673.732252] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 675.027897] Lustre: 39972:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 677.753637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 680.276725] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 684.241193] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 686.693631] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 695.740623] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 697.712735] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 697.714918] Lustre: Skipped 1 previous similar message [ 701.923985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 701.926524] Lustre: Skipped 15 previous similar messages [ 713.183129] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 713.274360] Lustre: server umount lustre-MDT0000 complete [ 715.031684] LustreError: 38838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086621 with bad export cookie 9866462866709444924 [ 715.035414] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 715.036963] LustreError: 38838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 721.206814] Lustre: server umount lustre-MDT0001 complete [ 724.888787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 725.407552] LustreError: 3634:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8f6690884e00 x1866555448421888/t0(0) o250->MGC192.168.201.160@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 727.213242] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 730.670813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 732.487027] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 734.860331] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 745.446574] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 749.544596] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 749.547481] Lustre: Skipped 1 previous similar message [ 751.036197] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 15 sec [ 752.408837] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 753.072209] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 756.704857] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 756.707770] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 756.711930] Lustre: Skipped 7 previous similar messages [ 760.860519] Lustre: server umount lustre-MDT0000 complete [ 762.338200] LustreError: 39591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086668 with bad export cookie 9866462866709446016 [ 762.339660] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 762.343490] LustreError: 39591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 762.461212] Lustre: server umount lustre-MDT0001 complete [ 774.149567] Lustre: server umount lustre-OST0000 complete [ 785.841729] Lustre: server umount lustre-OST0001 complete [ 789.447441] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:31:35 (1780086695) [ 795.791416] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 799.830749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 799.995518] LustreError: 46191:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 800.002765] LustreError: 46191:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 801.475437] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 804.773325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 806.483524] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 807.583763] Lustre: 47310:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 807.587454] Lustre: 47310:0:(mgs_llog.c:1437:mgs_modify_param()) Skipped 1 previous similar message [ 810.234662] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 812.608187] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 816.189052] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 818.838090] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 827.781406] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 829.589677] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 840.578969] Lustre: server umount lustre-MDT0000 complete [ 842.157531] LustreError: 46173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780086748 with bad export cookie 9866462866709446842 [ 842.161961] LustreError: 46173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 842.164548] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 842.325710] Lustre: server umount lustre-MDT0001 complete [ 853.512155] Lustre: server umount lustre-OST0000 complete [ 865.191686] Lustre: server umount lustre-OST0001 complete [ 869.154145] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:32:55 (1780086775) [ 870.275135] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 870.990984] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:32:56 (1780086776) [ 875.021322] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 878.338042] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 888.518358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 888.663834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 888.712150] Lustre: lustre-MDT0000: new disk, initializing [ 888.754107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 890.586254] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 894.439974] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 897.433407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 897.559940] Lustre: 53191:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 898.960480] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 898.977814] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 899.861193] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 903.328430] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 904.187743] Lustre: DEBUG MARKER: create test files [ 919.520667] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 922.640476] Lustre: server umount lustre-OST0000 complete [ 924.256845] Lustre: server umount lustre-MDT0000 complete [ 928.742915] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 930.101155] Key type lgssc unregistered [ 930.256947] LNet: 54686:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 930.260205] LNetError: 54686:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 930.269393] LNet: Removed LNI 192.168.201.160@tcp [ 930.721577] Key type .llcrypt unregistered [ 930.723912] Key type ._llcrypt unregistered [ 943.224987] Key type ._llcrypt registered [ 943.226189] Key type .llcrypt registered [ 943.275245] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 943.660955] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 943.711772] alg: No test for adler32 (adler32-zlib) [ 944.587845] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 944.693831] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 946.295299] Key type lgssc registered [ 946.774743] Lustre: Echo OBD driver; http://www.lustre.org/ [ 949.805219] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 949.814664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 951.033876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 952.742215] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 955.005950] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 957.889539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 958.046482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 960.666510] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 961.132906] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 961.166791] Lustre: 57277:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 961.171233] Lustre: 57277:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 961.174498] Lustre: 57277:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 961.177772] Lustre: 57277:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 961.180757] Lustre: 57277:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 961.183704] Lustre: 57277:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 963.019089] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 966.302136] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 968.909765] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 971.744734] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 971.745217] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 971.757847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 976.864717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 977.409226] Lustre: server umount lustre-OST0000 complete [ 979.407672] Lustre: server umount lustre-MDT0000 complete [ 983.749086] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 984.901956] Key type lgssc unregistered [ 985.051603] LNet: 58755:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 985.054419] LNetError: 58755:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 985.067842] LNet: Removed LNI 192.168.201.160@tcp [ 985.438073] Key type .llcrypt unregistered [ 985.439758] Key type ._llcrypt unregistered [ 998.753665] Key type ._llcrypt registered [ 998.756241] Key type .llcrypt registered [ 998.803133] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 999.243598] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 999.264433] alg: No test for adler32 (adler32-zlib) [ 1000.165593] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 1000.290145] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 1001.903186] Key type lgssc registered [ 1002.458628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1006.266038] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1006.277152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1007.534107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1009.441915] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1012.047645] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1015.347390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1015.534319] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1017.572887] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1017.604941] Lustre: 61436:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1017.607995] Lustre: 61436:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1017.610572] Lustre: 61436:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1017.613917] Lustre: 61436:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1017.619173] Lustre: 61436:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1017.622538] Lustre: 61436:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1018.290939] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1020.646333] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1023.960350] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1028.065527] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1028.066285] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1028.075209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1032.929455] Lustre: server umount lustre-OST0000 complete [ 1034.552499] Lustre: server umount lustre-MDT0000 complete [ 1038.789095] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 1039.978858] Key type lgssc unregistered [ 1040.148048] LNet: 62863:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1040.154988] LNetError: 62863:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1040.168990] LNet: Removed LNI 192.168.201.160@tcp [ 1040.521046] Key type .llcrypt unregistered [ 1040.522492] Key type ._llcrypt unregistered [ 1048.641374] Key type ._llcrypt registered [ 1048.642874] Key type .llcrypt registered [ 1048.703057] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1055.936877] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1056.322865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1056.377335] alg: No test for adler32 (adler32-zlib) [ 1057.289827] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 1057.402437] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 1058.999220] Key type lgssc registered [ 1059.481352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1073.164880] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1073.171612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1074.265354] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1074.281932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1074.321051] Lustre: lustre-MDT0000: new disk, initializing [ 1074.350943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1074.359425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1075.812648] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1080.385528] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1080.429844] Lustre: 65956:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1080.455949] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1080.458347] Lustre: Skipped 1 previous similar message [ 1080.498124] Lustre: lustre-MDT0001: new disk, initializing [ 1080.531839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1080.544130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1080.548355] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1081.990129] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1085.371731] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1086.724365] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1089.429491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1089.558153] Lustre: lustre-OST0000: new disk, initializing [ 1089.560579] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1089.563917] Lustre: 67698:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1089.606104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1090.753837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1090.758789] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1090.816284] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1091.866813] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1095.297587] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1096.920189] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1097.030683] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1098.626110] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1098.716217] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1101.279961] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1101.283454] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1101.286580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1101.287472] Lustre: Skipped 1 previous similar message [ 1101.291200] Lustre: Skipped 1 previous similar message [ 1105.552192] Lustre: server umount lustre-OST0000 complete [ 1111.520710] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1111.523050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1111.525334] Lustre: Skipped 1 previous similar message [ 1111.528506] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1111.531537] LustreError: Skipped 1 previous similar message [ 1116.640218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1116.642448] Lustre: Skipped 2 previous similar messages [ 1120.736363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1120.738303] Lustre: Skipped 1 previous similar message [ 1121.247142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1121.311974] Lustre: server umount lustre-MDT0000 complete [ 1122.720120] LustreError: 65949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087029 with bad export cookie 7012296493900206272 [ 1122.722526] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1122.724090] LustreError: 65949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1122.838584] Lustre: server umount lustre-MDT0001 complete [ 1126.009730] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:37:12 (1780087032) [ 1129.748935] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1144.107990] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1147.231276] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1159.209746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1159.295213] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1159.308707] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1159.350968] Lustre: lustre-MDT0000: new disk, initializing [ 1159.381461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1159.387994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1160.787579] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1165.330481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1165.373458] Lustre: 73509:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1165.395989] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1165.398251] Lustre: Skipped 1 previous similar message [ 1165.434785] Lustre: lustre-MDT0001: new disk, initializing [ 1165.470435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1165.483626] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1165.489139] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1167.012023] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1170.424176] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1171.962363] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1174.542369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1174.663495] Lustre: lustre-OST0000: new disk, initializing [ 1174.665720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1174.669532] Lustre: 75252:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1174.701836] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1176.787454] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1180.190571] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1181.804789] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1184.766176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1184.769865] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1184.798590] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1184.951972] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1186.522882] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1186.602651] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1189.856558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1189.859393] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1189.864575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1189.866598] Lustre: Skipped 1 previous similar message [ 1193.437515] Lustre: server umount lustre-OST0000 complete [ 1194.977660] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1194.982685] Lustre: Skipped 2 previous similar messages [ 1200.096416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1200.098822] Lustre: Skipped 3 previous similar messages [ 1209.311403] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1209.405196] Lustre: server umount lustre-MDT0000 complete [ 1211.196651] LustreError: 73500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087117 with bad export cookie 7012296493900207147 [ 1211.197452] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1211.209480] LustreError: 73500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1211.327625] Lustre: server umount lustre-MDT0001 complete [ 1214.813668] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:38:40 (1780087120) [ 1217.417408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1217.593493] LustreError: 77536:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1217.599805] LustreError: 77536:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1217.616828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1218.969549] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1222.226798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1223.917504] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1226.035282] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1227.347233] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1229.889400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1232.110224] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1233.916793] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1235.174560] Lustre: 79863:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 56275d27-683e-4b7a-b17e-73013b660039 at adminstrative request [ 1237.474969] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1237.481083] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1247.713985] Lustre: MGS: Client 56275d27-683e-4b7a-b17e-73013b660039 (at 0@lo) reconnecting [ 1247.717978] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x6150ae822aa3978f to 0x6150ae822aa39a1a [ 1247.723608] Lustre: MGC192.168.201.160@tcp: Connection restored to 0@lo (at 0@lo) [ 1267.710356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1267.784130] Lustre: lustre-OST0001: new disk, initializing [ 1267.786242] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1267.789116] Lustre: 80259:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1267.827491] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1267.832507] Lustre: Skipped 2 previous similar messages [ 1269.845852] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1273.172678] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1274.239095] Lustre: server umount lustre-OST0001 complete [ 1277.941119] LustreError: 79876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1277.944292] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1277.952069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1277.954413] Lustre: Skipped 3 previous similar messages [ 1281.678421] Lustre: server umount lustre-OST0000 complete [ 1285.082229] Lustre: server umount lustre-MDT0000 complete [ 1286.433050] LustreError: 77518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087193 with bad export cookie 7012296493900208666 [ 1286.435850] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1286.436790] LustreError: 77518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1292.623057] Lustre: server umount lustre-MDT0001 complete [ 1295.315509] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:40:01 (1780087201) [ 1295.814715] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1296.365316] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:40:02 (1780087202) [ 1296.927728] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1297.564155] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:40:03 (1780087203) [ 1298.094111] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1298.648489] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:40:04 (1780087204) [ 1299.140417] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1299.685461] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:40:05 (1780087205) [ 1300.200650] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1302.121265] loop: module loaded [ 1302.125376] loop0: detected capacity change from 0 to 4294967296 [ 1302.139541] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1302.644724] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1303.700885] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1305.012000] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1314.023148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1314.040349] LustreError: 84238:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1314.043046] LustreError: 84238:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1314.045353] LustreError: 84238:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1314.049365] LustreError: 84238:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1320.226281] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1323.364841] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1335.527726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1335.626862] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1335.638549] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1335.679485] Lustre: lustre-MDT0000: new disk, initializing [ 1335.707121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1335.714229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1336.999235] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1341.145225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1341.187280] Lustre: 87080:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1341.287831] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1342.624388] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1345.692413] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1346.843796] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1349.046871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1349.151701] Lustre: 88823:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1350.977571] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1353.987846] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1355.334087] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1355.768932] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1355.816676] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1356.431953] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1357.705827] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1357.774719] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1360.866439] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1360.866970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1360.869918] LustreError: Skipped 1 previous similar message [ 1360.874267] Lustre: Skipped 3 previous similar messages [ 1360.877728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1360.879973] Lustre: Skipped 3 previous similar messages [ 1364.492147] Lustre: server umount lustre-OST0000 complete [ 1371.804533] Lustre: server umount lustre-MDT0000 complete [ 1373.012125] LustreError: 87073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087279 with bad export cookie 7012296493900209247 [ 1373.014689] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1373.015432] LustreError: 87073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1373.120025] Lustre: server umount lustre-MDT0001 complete [ 1375.796132] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:41:21 (1780087281) [ 1377.866633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1378.021160] LustreError: 91089:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1378.026694] LustreError: 91089:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1379.207675] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1381.949289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1383.335852] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1385.067332] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1386.239647] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1388.377415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1390.230275] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1391.912725] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1393.113428] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1393.169622] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1394.375716] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1394.447333] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1396.830980] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1396.905969] Lustre: 94081:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1398.893413] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1401.955830] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1403.192020] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1404.413951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1404.416271] Lustre: Skipped 2 previous similar messages [ 1404.417745] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1404.441964] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1405.293809] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1406.574380] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1406.642819] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1416.741632] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1422.350189] Lustre: Failing over lustre-MDT0000 [ 1422.655422] Lustre: server umount lustre-MDT0000 complete [ 1423.328460] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1423.331120] LustreError: Skipped 1 previous similar message [ 1423.333051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1423.337866] Lustre: Skipped 3 previous similar messages [ 1423.340440] LustreError: 91090:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1428.693148] LustreError: 94913:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1428.699204] LustreError: 94913:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 1437.327709] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1437.329836] LDISKFS-fs (dm-0): recovery complete [ 1437.332959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1437.370234] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1437.477272] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1438.695430] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1438.933910] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1438.934215] Lustre: 91085:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1442.788784] Lustre: 91800:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1442.792801] Lustre: 91800:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 1442.794890] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1442.818604] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1443.135143] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1443.449635] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1443.453775] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 1443.767194] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1443.770098] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 1444.089722] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1444.094892] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 8 previous similar messages [ 1444.721726] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1444.725824] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 3 previous similar messages [ 1445.039157] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1445.042641] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 3 previous similar messages [ 1446.296090] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1446.299455] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 6 previous similar messages [ 1446.928153] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1446.931313] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 1447.247134] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1447.250753] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 1450.377146] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1450.380876] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 1451.008895] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1451.012588] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 1451.327148] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1451.329991] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 1458.513542] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1458.517691] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 25 previous similar messages [ 1459.145685] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1459.148555] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 1459.463118] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1459.465712] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 1474.761477] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1474.765700] Lustre: 96769:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 51 previous similar messages [ 1475.393913] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1475.397549] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 51 previous similar messages [ 1475.711102] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1475.714219] LustreError: 96769:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 51 previous similar messages [ 1498.183154] Lustre: 96769:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1498.187105] Lustre: 96769:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1498.190447] Lustre: 96769:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client e841f748-ee74-4a78-8277-938be46cf9fd@192.168.201.60@tcp [ 1498.194855] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1498.197403] Lustre: 96769:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@00000000061a13f8 x1866556460041728/t0(8589934774) o35->e841f748-ee74-4a78-8277-938be46cf9fd@192.168.201.60@tcp:510/0 lens 392/0 e 0 to 0 dl 1780087415 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1498.207734] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1498.214092] Lustre: lustre-MDT0000: Denying connection for new client e841f748-ee74-4a78-8277-938be46cf9fd (at 192.168.201.60@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1498.217187] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1498.239775] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1498.240308] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1498.244344] Lustre: Skipped 2 previous similar messages [ 1498.260678] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1498.260837] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1503.712613] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1503.714868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1503.717871] Lustre: Skipped 3 previous similar messages [ 1503.719575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1503.721020] Lustre: Skipped 4 previous similar messages [ 1504.445855] Lustre: server umount lustre-OST0000 complete [ 1509.344736] LustreError: 92682:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1509.349497] LustreError: 92682:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 1520.095120] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1520.150031] Lustre: server umount lustre-OST0001 complete [ 1524.191950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1524.195447] LustreError: Skipped 3 previous similar messages [ 1527.587159] Lustre: server umount lustre-MDT0000 complete [ 1528.947288] LustreError: 91071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087435 with bad export cookie 7012296493900260431 [ 1528.950219] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1528.951314] LustreError: 91071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1529.078847] Lustre: server umount lustre-MDT0001 complete [ 1531.767311] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:43:57 (1780087437) [ 1534.248149] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1534.251808] Lustre: *** cfs_fail_loc=197, val=0*** [ 1549.663505] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1554.784511] LustreError: 98721:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.160@tcp: failed processing log, type 4: rc = -110 [ 1559.910150] Lustre: *** cfs_fail_loc=197, val=0*** [ 1559.911205] Lustre: Skipped 65 previous similar messages [ 1580.384374] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1580.386509] Lustre: Skipped 7 previous similar messages [ 1582.212789] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1583.885474] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1584.558603] Lustre: server umount lustre-OST0000 complete [ 1587.204982] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:44:53 (1780087493) [ 1600.880726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1601.064133] LustreError: 100225:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1601.070859] LustreError: 100225:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1602.429761] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1605.502889] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1607.468913] LustreError: 100257:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1607.473650] LustreError: 100257:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1607.477936] LustreError: 100257:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1607.599048] Lustre: server umount lustre-MDT0000 complete [ 1618.174457] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1621.195686] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1633.603251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1633.698632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1633.709132] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1633.711123] Lustre: Skipped 4 previous similar messages [ 1633.746837] Lustre: lustre-MDT0000: new disk, initializing [ 1633.748290] Lustre: Skipped 3 previous similar messages [ 1633.777235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1635.083180] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1639.302262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1639.349230] Lustre: 104274:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1639.459601] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1640.793916] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1643.998322] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1645.187506] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1647.441397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1647.541608] Lustre: 106020:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1649.399983] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1651.476151] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1652.387332] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1653.771207] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1653.839832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1655.164486] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1655.244342] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1656.800663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1656.802345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1656.802619] LustreError: Skipped 1 previous similar message [ 1656.805353] Lustre: Skipped 5 previous similar messages [ 1656.806854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1656.808598] Lustre: Skipped 13 previous similar messages [ 1658.006206] Lustre: server umount lustre-OST0000 complete [ 1665.366138] Lustre: server umount lustre-MDT0000 complete [ 1666.563146] LustreError: 106019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087573 with bad export cookie 7012296493900271057 [ 1666.565615] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1666.566803] LustreError: 106019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1666.670493] Lustre: server umount lustre-MDT0001 complete [ 1669.478992] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:46:15 (1780087575) [ 1672.124236] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 1673.037427] Key type lgssc unregistered [ 1673.154336] LNet: 108335:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1673.156543] LNetError: 108335:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1673.165265] LNet: Removed LNI 192.168.201.160@tcp [ 1673.429160] Key type .llcrypt unregistered [ 1673.430065] Key type ._llcrypt unregistered [ 1678.836253] Key type ._llcrypt registered [ 1678.837162] Key type .llcrypt registered [ 1678.872136] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1684.554039] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1684.898940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1684.954060] alg: No test for adler32 (adler32-zlib) [ 1685.803139] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 1685.886611] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 1687.471184] Key type lgssc registered [ 1687.838723] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1700.236750] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1700.243876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1701.332463] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1701.342572] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1701.379880] Lustre: lustre-MDT0000: new disk, initializing [ 1701.405059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1701.412877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1702.606570] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1706.439369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1706.551132] Lustre: lustre-OST0000: new disk, initializing [ 1706.553074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1706.554507] Lustre: Skipped 1 previous similar message [ 1706.556207] Lustre: 112430:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1706.576606] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1706.876703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1706.880561] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1706.891081] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1708.292995] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1712.162679] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1712.217294] Lustre: lustre-OST0001: new disk, initializing [ 1712.218738] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1712.220591] Lustre: 113276:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1712.241807] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1713.992540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1713.995149] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1714.004722] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1714.008576] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1719.264466] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1719.265111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1719.268266] Lustre: Skipped 1 previous similar message [ 1724.006570] Lustre: server umount lustre-MDT0000 complete [ 1725.935645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1727.368280] LustreError: 111632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087633 with bad export cookie 7354259981155298023 [ 1727.370385] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1727.371214] LustreError: 111632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1727.401838] Lustre: server umount lustre-OST0000 complete [ 1728.653354] Lustre: server umount lustre-OST0001 complete [ 1731.543619] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 16:47:17 (1780087637) [ 1734.741371] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1741.770412] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:47:27 (1780087647) [ 1753.633271] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 1756.536668] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1767.238864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1767.339618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1767.353857] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1767.389890] Lustre: lustre-MDT0000: new disk, initializing [ 1767.415442] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1767.422593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1768.619755] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1772.484572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1772.524343] Lustre: 119940:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1772.548635] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1772.550104] Lustre: Skipped 1 previous similar message [ 1772.583442] Lustre: lustre-MDT0001: new disk, initializing [ 1772.615933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1772.626121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1772.629946] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1773.837684] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1776.775465] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1777.879646] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1780.031266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1780.134949] Lustre: 121681:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1781.496602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1781.499344] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1781.529175] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1781.875410] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1784.907469] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1786.225793] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1786.300901] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1787.553469] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1787.628657] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1791.970050] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1791.970797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1791.973055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1791.975657] Lustre: Skipped 1 previous similar message [ 1791.980674] Lustre: Skipped 2 previous similar messages [ 1794.383868] Lustre: server umount lustre-OST0000 complete [ 1797.089159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1797.089843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1797.092656] Lustre: Skipped 1 previous similar message [ 1797.097509] Lustre: Skipped 1 previous similar message [ 1801.821235] Lustre: server umount lustre-MDT0000 complete [ 1803.051265] LustreError: 119934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087709 with bad export cookie 7354259981155299234 [ 1803.054303] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1803.055224] LustreError: 119934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1803.195329] Lustre: server umount lustre-MDT0001 complete [ 1806.006585] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:48:32 (1780087712) [ 1808.341187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1808.511511] LustreError: 123941:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1808.520709] LustreError: 123941:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1808.537858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1808.541523] Lustre: Skipped 1 previous similar message [ 1809.894333] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1812.898591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1814.386031] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1816.091632] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1817.185676] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1819.375154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1821.326225] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1823.051409] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1840.799358] Lustre: *** cfs_fail_loc=159, val=0*** [ 1857.253306] Lustre: lustre-MDT0000: Client 298ff44d-002b-47b5-b4fb-ad7aa9723fa1 (at 192.168.201.60@tcp) reconnecting [ 1857.784801] Lustre: *** cfs_fail_loc=159, val=0*** [ 1857.786344] Lustre: Skipped 5 previous similar messages [ 1874.146375] Lustre: lustre-MDT0000: Client 298ff44d-002b-47b5-b4fb-ad7aa9723fa1 (at 192.168.201.60@tcp) reconnecting [ 1889.247773] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1889.249706] LustreError: Skipped 1 previous similar message [ 1889.250833] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1889.254569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1890.273100] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1895.056831] Lustre: server umount lustre-OST0000 complete [ 1899.999904] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1900.002140] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1900.005609] Lustre: Skipped 1 previous similar message [ 1900.008044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1900.009533] Lustre: Skipped 2 previous similar messages [ 1902.420928] Lustre: server umount lustre-MDT0000 complete [ 1903.562181] LustreError: 123923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087810 with bad export cookie 7354259981155300102 [ 1903.564424] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1903.566146] LustreError: 123923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1903.684252] Lustre: server umount lustre-MDT0001 complete [ 1906.964666] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 1907.900610] Key type lgssc unregistered [ 1908.022352] LNet: 127380:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1908.025094] LNetError: 127380:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1908.036342] LNet: Removed LNI 192.168.201.160@tcp [ 1908.317097] Key type .llcrypt unregistered [ 1908.317988] Key type ._llcrypt unregistered [ 1916.909706] Key type ._llcrypt registered [ 1916.910641] Key type .llcrypt registered [ 1916.954627] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 1917.261887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1917.268916] alg: No test for adler32 (adler32-zlib) [ 1918.121864] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 1918.202879] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 1919.791178] Key type lgssc registered [ 1920.135353] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1922.646746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1922.653199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1923.825301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1925.076314] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1927.737685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1927.857075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1929.117401] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1930.833609] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1932.068172] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1934.216177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1934.305387] LustreError: 130350:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1934.310699] LustreError: 130350:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1934.343608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1936.076464] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 1937.447663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1937.665318] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1943.112522] Lustre: *** cfs_fail_loc=159, val=0*** [ 1958.624841] Lustre: lustre-MDT0000: Client 2e9426af-f6f4-42a7-8208-65e469589331 (at 192.168.201.60@tcp) reconnecting [ 1959.322069] Lustre: *** cfs_fail_loc=159, val=0*** [ 1974.492686] Lustre: lustre-MDT0001: Client 2e9426af-f6f4-42a7-8208-65e469589331 (at 192.168.201.60@tcp) reconnecting [ 1974.984273] Lustre: *** cfs_fail_loc=159, val=0*** [ 1974.985891] Lustre: Skipped 3 previous similar messages [ 1991.393546] Lustre: lustre-MDT0001: Client 2e9426af-f6f4-42a7-8208-65e469589331 (at 192.168.201.60@tcp) reconnecting [ 1992.549502] Lustre: 131451:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1993.333437] Lustre: *** cfs_fail_loc=159, val=0*** [ 1993.335312] Lustre: Skipped 4 previous similar messages [ 2008.800198] Lustre: lustre-MDT0000: Client 71bd1e49-c39c-40fb-a102-0d8726adbcaf (at 192.168.201.60@tcp) reconnecting [ 2009.329359] Lustre: *** cfs_fail_loc=159, val=0*** [ 2009.330805] Lustre: Skipped 14 previous similar messages [ 2024.672240] Lustre: lustre-MDT0000: Client 71bd1e49-c39c-40fb-a102-0d8726adbcaf (at 192.168.201.60@tcp) reconnecting [ 2025.224427] Lustre: 131740:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2029.537234] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2029.538416] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2029.538989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2029.539588] LustreError: Skipped 1 previous similar message [ 2029.542435] Lustre: Skipped 1 previous similar message [ 2032.592891] Lustre: server umount lustre-OST0000 complete [ 2034.656594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2034.661376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2034.662794] Lustre: Skipped 2 previous similar messages [ 2039.776213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2039.777826] Lustre: Skipped 1 previous similar message [ 2040.090990] Lustre: server umount lustre-MDT0000 complete [ 2041.264221] LustreError: 128736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780087947 with bad export cookie 226614132835777597 [ 2041.267322] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2041.268213] LustreError: 128736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2041.381866] Lustre: server umount lustre-MDT0001 complete [ 2044.634982] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2045.601317] Key type lgssc unregistered [ 2045.729356] LNet: 132772:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2045.731352] LNetError: 132772:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2045.738273] LNet: Removed LNI 192.168.201.160@tcp [ 2046.030278] Key type .llcrypt unregistered [ 2046.031203] Key type ._llcrypt unregistered [ 2054.293054] Key type ._llcrypt registered [ 2054.293979] Key type .llcrypt registered [ 2054.335935] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2054.617040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2054.629893] alg: No test for adler32 (adler32-zlib) [ 2055.472279] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2055.542819] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2057.119150] Key type lgssc registered [ 2057.459822] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2059.987310] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2059.994163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2061.164733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2062.363335] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2065.042065] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2065.164413] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2066.430912] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2068.060957] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2069.137242] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2071.236091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2071.359314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2072.426980] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2073.134836] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2074.472878] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2074.722852] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2077.862131] Lustre: 136389:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2078.537804] Lustre: 136437:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2079.016336] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2080.216340] Lustre: 136583:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2080.545529] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2080.548670] Lustre: Skipped 1 previous similar message [ 2080.853762] Lustre: 136681:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2082.784057] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2082.786275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2082.790700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2084.832740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2084.833185] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2084.837273] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2087.954651] Lustre: server umount lustre-OST0000 complete [ 2089.952448] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2089.952943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2089.958899] Lustre: Skipped 1 previous similar message [ 2089.960884] Lustre: Skipped 1 previous similar message [ 2095.072758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2095.075939] Lustre: Skipped 2 previous similar messages [ 2095.315337] Lustre: server umount lustre-MDT0000 complete [ 2096.585348] LustreError: 134126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088003 with bad export cookie 13455421273832756856 [ 2096.589249] LustreError: 134126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2096.590030] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2096.714711] Lustre: server umount lustre-MDT0001 complete [ 2100.577033] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2101.707223] Key type lgssc unregistered [ 2101.843586] LNet: 137719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2101.847827] LNetError: 137719:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2101.858429] LNet: Removed LNI 192.168.201.160@tcp [ 2102.170180] Key type .llcrypt unregistered [ 2102.171585] Key type ._llcrypt unregistered [ 2111.217508] Key type ._llcrypt registered [ 2111.219377] Key type .llcrypt registered [ 2111.262719] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2111.634307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2111.640704] alg: No test for adler32 (adler32-zlib) [ 2112.500545] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2112.590136] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2114.175217] Key type lgssc registered [ 2114.543530] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2117.266586] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2117.273343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2118.453908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2119.863398] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2122.800976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2122.942746] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2124.277116] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2126.062443] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2127.217851] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2129.482761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2129.603522] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2131.507170] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2132.710961] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2133.365428] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2143.970303] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2147.199624] Lustre: *** cfs_fail_loc=159, val=0*** [ 2163.425910] Lustre: lustre-MDT0000: Client b81c2d6c-b7e0-457a-89b0-7439fc273c9a (at 192.168.201.60@tcp) reconnecting [ 2179.039867] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2179.043235] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2179.049419] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2180.064505] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2180.066176] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2180.067200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2180.072407] Lustre: Skipped 1 previous similar message [ 2184.335122] Lustre: server umount lustre-OST0000 complete [ 2189.791837] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2189.795340] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2189.799789] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2191.697618] Lustre: server umount lustre-MDT0000 complete [ 2192.904306] LustreError: 141385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088099 with bad export cookie 11921469955405731528 [ 2192.907337] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2192.909031] LustreError: 141385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2193.014473] Lustre: server umount lustre-MDT0001 complete [ 2196.580772] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2197.664697] Key type lgssc unregistered [ 2197.787461] LNet: 142421:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2197.791158] LNetError: 142421:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2197.798382] LNet: Removed LNI 192.168.201.160@tcp [ 2198.095210] Key type .llcrypt unregistered [ 2198.096483] Key type ._llcrypt unregistered [ 2207.260959] Key type ._llcrypt registered [ 2207.262015] Key type .llcrypt registered [ 2207.304376] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2207.619680] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2207.626267] alg: No test for adler32 (adler32-zlib) [ 2208.491234] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2208.586424] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2210.167134] Key type lgssc registered [ 2210.527877] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2213.196496] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2213.203549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2214.375356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2215.675498] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2218.448062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2218.579303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2219.849572] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2221.682048] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2222.934567] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2225.252940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2225.392540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2227.362140] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2228.454989] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2229.184936] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2239.719745] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2257.671336] Lustre: lustre-OST0000: evicting 81530d54-0b7b-4bff-83f3-9a699c85a26f (at 192.168.201.60@tcp) by administrative request [ 2257.674617] Lustre: Skipped 1 previous similar message [ 2285.024781] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2285.027027] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2285.030371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.050731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.051797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2286.052267] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2291.169461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2291.172828] Lustre: Skipped 2 previous similar messages [ 2296.287114] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2296.288515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2296.291819] Lustre: Skipped 1 previous similar message [ 2296.342639] Lustre: server umount lustre-OST0000 complete [ 2300.383919] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2300.388062] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2300.394319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2303.769930] Lustre: server umount lustre-MDT0000 complete [ 2305.057285] LustreError: 145420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088211 with bad export cookie 1707582703348171934 [ 2305.060730] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2305.061892] LustreError: 145420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2305.193746] Lustre: server umount lustre-MDT0001 complete [ 2308.862489] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2309.822439] Key type lgssc unregistered [ 2309.954316] LNet: 147532:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2309.956377] LNetError: 147532:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2309.962288] LNet: Removed LNI 192.168.201.160@tcp [ 2310.246207] Key type .llcrypt unregistered [ 2310.247966] Key type ._llcrypt unregistered [ 2334.603745] Key type ._llcrypt registered [ 2334.604693] Key type .llcrypt registered [ 2334.645469] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2334.974611] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2334.991846] alg: No test for adler32 (adler32-zlib) [ 2335.845493] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2335.935942] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2337.519158] Key type lgssc registered [ 2337.876657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2340.350658] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2340.356504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2341.507995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2342.838403] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2345.605696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2345.727281] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2346.988119] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2348.783878] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2349.984752] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2352.258815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2352.372927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2354.281871] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2356.037368] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2356.453200] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2366.696075] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2377.185029] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2377.185806] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2377.187111] LustreError: Skipped 1 previous similar message [ 2377.190756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2377.193130] Lustre: Skipped 1 previous similar message [ 2380.496195] Lustre: server umount lustre-OST0000 complete [ 2381.791800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2381.794555] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2381.798738] Lustre: Skipped 1 previous similar message [ 2381.800350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2387.423915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2387.425331] Lustre: Skipped 3 previous similar messages [ 2392.544741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2392.546766] Lustre: Skipped 1 previous similar message [ 2396.127086] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2396.194448] Lustre: server umount lustre-MDT0000 complete [ 2396.640348] LustreError: 149977:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2396.646026] LustreError: 149977:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2397.464568] LustreError: 149955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088304 with bad export cookie 4909859023349694400 [ 2397.468620] LustreError: 149955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2397.468879] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2397.592915] Lustre: server umount lustre-MDT0001 complete [ 2401.282131] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2402.333100] Key type lgssc unregistered [ 2402.470444] LNet: 153180:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2402.473370] LNetError: 153180:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2402.481339] LNet: Removed LNI 192.168.201.160@tcp [ 2402.782219] Key type .llcrypt unregistered [ 2402.783345] Key type ._llcrypt unregistered [ 2412.205915] Key type ._llcrypt registered [ 2412.207644] Key type .llcrypt registered [ 2412.249586] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2412.635318] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2412.677714] alg: No test for adler32 (adler32-zlib) [ 2413.527612] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2413.612317] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2415.191118] Key type lgssc registered [ 2415.544265] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2420.314876] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 16:58:46 (1780088326) [ 2425.562545] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 2428.838640] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2440.957677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2440.963730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2442.077572] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2442.090400] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2442.128527] Lustre: lustre-MDT0000: new disk, initializing [ 2442.157141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2442.164972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2443.526126] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2447.681994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2447.725339] Lustre: 157360:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2447.748654] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2447.750814] Lustre: Skipped 1 previous similar message [ 2447.783418] Lustre: lustre-MDT0001: new disk, initializing [ 2447.814772] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2447.832266] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2447.836174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2449.135233] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2452.120410] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2453.246274] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2455.367780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2455.462658] Lustre: lustre-OST0000: new disk, initializing [ 2455.464415] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2455.467101] Lustre: 159100:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2455.493642] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2456.630286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2456.633940] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2456.663185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2457.251054] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2460.281874] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2466.785968] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2466.786281] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2466.787489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2466.787787] LustreError: Skipped 1 previous similar message [ 2466.792065] Lustre: Skipped 1 previous similar message [ 2469.517437] Lustre: server umount lustre-OST0000 complete [ 2471.904695] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.905377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2471.909590] Lustre: Skipped 1 previous similar message [ 2471.911310] Lustre: Skipped 1 previous similar message [ 2477.024040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2477.025839] Lustre: Skipped 2 previous similar messages [ 2482.144258] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2482.146359] Lustre: Skipped 1 previous similar message [ 2485.215084] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2485.275218] Lustre: server umount lustre-MDT0000 complete [ 2486.452069] LustreError: 157351:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088393 with bad export cookie 10979455994301033765 [ 2486.454806] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2486.455180] LustreError: 157351:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2486.577156] Lustre: server umount lustre-MDT0001 complete [ 2489.906358] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2490.931111] Key type lgssc unregistered [ 2491.058399] LNet: 160874:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2491.060144] LNetError: 160874:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2491.070283] LNet: Removed LNI 192.168.201.160@tcp [ 2491.369401] Key type .llcrypt unregistered [ 2491.370506] Key type ._llcrypt unregistered [ 2499.851701] Key type ._llcrypt registered [ 2499.852510] Key type .llcrypt registered [ 2499.889723] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2500.178346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2500.192407] alg: No test for adler32 (adler32-zlib) [ 2501.044200] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2501.117236] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2502.687184] Key type lgssc registered [ 2503.011360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2505.481305] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2505.488228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2506.644610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2507.869179] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2510.692275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2510.833662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2512.287131] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2514.213125] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2515.341605] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2517.566527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2517.696877] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2519.625275] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 2521.523398] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2532.070846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2539.441206] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2539.447995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2539.451770] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2539.454175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.457463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2539.795286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.126237] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2540.128307] Lustre: Skipped 1 previous similar message [ 2540.133623] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2540.138935] Lustre: Skipped 1 previous similar message [ 2540.141496] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2540.143856] Lustre: Skipped 1 previous similar message [ 2540.145206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.148331] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2540.151386] Lustre: Skipped 1 previous similar message [ 2540.454044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.763601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.064399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.371091] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2541.373640] Lustre: Skipped 3 previous similar messages [ 2541.379141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2541.383080] Lustre: Skipped 3 previous similar messages [ 2541.384258] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2541.386869] Lustre: Skipped 3 previous similar messages [ 2541.388927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.392656] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2541.394281] Lustre: Skipped 3 previous similar messages [ 2541.677620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.964647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.255794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.576278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.852923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.133497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.418814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2543.420923] Lustre: Skipped 6 previous similar messages [ 2543.427647] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2543.432478] Lustre: Skipped 6 previous similar messages [ 2543.434140] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2543.435597] Lustre: Skipped 6 previous similar messages [ 2543.436771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.440498] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2543.442434] Lustre: Skipped 6 previous similar messages [ 2543.730635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.005946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.282170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.562347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.853102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.140306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.429456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.717508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.010139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.289974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.570842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.861507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.166767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.461996] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2547.463584] Lustre: Skipped 13 previous similar messages [ 2547.468464] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2547.472981] Lustre: Skipped 13 previous similar messages [ 2547.474598] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2547.476838] Lustre: Skipped 13 previous similar messages [ 2547.478595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.482893] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2547.484855] Lustre: Skipped 13 previous similar messages [ 2547.768514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.053918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.350933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.633530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.920763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.200605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.489962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.782815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.079589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.362407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.660581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.945037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.226292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.508194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.812613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.090625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.367198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.644380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.932953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.200198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.483225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.790545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.109595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.409631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.710581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.012738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.321815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.613225] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2555.616559] Lustre: Skipped 27 previous similar messages [ 2555.623295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2555.627083] Lustre: Skipped 27 previous similar messages [ 2555.628537] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2555.630696] Lustre: Skipped 27 previous similar messages [ 2555.632651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.635874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2555.638597] Lustre: Skipped 27 previous similar messages [ 2555.914881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.205705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.515704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.814667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.114767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.397341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.695276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.981367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.265237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.548866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.847819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.127563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.415958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.705025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.995054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.276703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.560077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.846340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.155871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.443577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.740549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.025660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.330674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.634269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.926023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.210191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.520601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.813887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.104383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.383528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.702490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.997747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.292264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.591208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.905393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.197960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.504760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.796283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.110972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.399385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.708882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.009505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.319780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.612382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.903260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.189712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.484144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.781950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.089326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.371203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.665132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.953066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.241124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.532784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.831135] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2571.834028] Lustre: Skipped 54 previous similar messages [ 2571.840349] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2571.843095] Lustre: Skipped 54 previous similar messages [ 2571.844334] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2571.845757] Lustre: Skipped 54 previous similar messages [ 2571.846841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.851412] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2571.853119] Lustre: Skipped 54 previous similar messages [ 2572.136937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.436771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.743611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.053917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.360279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.660829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.958820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.261438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.542287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.836450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.131294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.440952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.736387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.044110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.353342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.656564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.932554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.217423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.512903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.834041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.133980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.424628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.717374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.022212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.312239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.604806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.898368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.200212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.483455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.790060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.078518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.380249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.658078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.949814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.238950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.540655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.833754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.111893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.391987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.689883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.978597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.262311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.543350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.845238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.142114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.429404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.706815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.980753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.253777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.529801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.803971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.085822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.360630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.638809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.917821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.208821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.475676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.763029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.039997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.319453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.595134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.862549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.135334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.420554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.695211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.971468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.247108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.534193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.821736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.109685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.390694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.675708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.947318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.215344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.484326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.755953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.023747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.298785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.583672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.887124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.161475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.438857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.714123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.001153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.283750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.564609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.864228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.153835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.433244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.721975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.989115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.266027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.531764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.799484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.077586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.383095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.684645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.997124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.305259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.628681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.931980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.250665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.545799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.870271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.175076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.481259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.785312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.114641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.417593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.726313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.036066] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2604.038302] Lustre: Skipped 110 previous similar messages [ 2604.044136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2604.048786] Lustre: Skipped 110 previous similar messages [ 2604.050282] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2604.052743] Lustre: Skipped 110 previous similar messages [ 2604.054525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.057735] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2604.059730] Lustre: Skipped 110 previous similar messages [ 2604.396731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.711611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.009219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.312933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.630188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.937953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.241543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.542578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.876736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.183282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.479144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.780859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.100461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.401164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.705149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.017904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.337533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.630737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.943033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.232038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.554623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.873231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.188119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.497146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.844837] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2611.848112] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2611.849065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.176785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.471466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.768579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.090048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.397952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.699952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.016686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.345946] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2614.349466] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2614.351190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.669835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.990155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.319994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.666250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.962268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.276565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.579649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.884232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.168561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.471113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.780917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.116806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.433363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.754261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.079247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.409309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.717312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.021185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.327710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.648665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.977978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.312098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.619349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.959333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.279267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.586880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.884844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.186753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.473409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.767637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.076417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.400630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.707748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.021381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.348240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.697705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.007302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.313031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.624066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.960037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.273814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.576662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.882622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.210339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.511257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.807952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.112793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.425133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.719279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.017367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.319639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.639022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.938815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.241974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.557041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.873070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.183101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.486835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.797334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.117272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.419300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.730886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.041120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.372075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.695150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.026745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.340173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.671488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.969319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.276373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.597953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.918103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.228507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.528296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.837837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.172026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.493680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.825310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.177879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.518504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.834369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.152364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.479607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.815653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.129356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.457080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.778645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.114785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.441706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.761044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.080507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.399730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.733272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.075165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.402111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.755900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.077739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.387136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.701804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.027993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.349711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.656673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.973624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.302146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.613205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.943958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.265912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.591387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.923250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.231257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.566053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.901677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.212296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.534571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.858268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.204941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.525811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.846869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.152401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.471542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.776319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.091175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.393271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.705238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.014677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.343637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.642666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.967328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.276935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.586397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.897129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.221321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.520970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.823072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.138210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.477190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.784249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.100929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.400591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.708684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.013951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.329715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.646462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.964235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.283390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.620689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.945220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.283147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.601054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.927942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.224881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.556571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.868982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.183505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.494335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.864513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.161503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.473850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.786523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.105833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.417229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.721759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.032415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.339092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.642764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.945978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.257846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.580472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.897879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.197599] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2668.200170] Lustre: Skipped 202 previous similar messages [ 2668.206023] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2668.209718] Lustre: Skipped 202 previous similar messages [ 2668.211239] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2668.214604] Lustre: Skipped 202 previous similar messages [ 2668.217284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.220462] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2668.222631] Lustre: Skipped 202 previous similar messages [ 2668.532224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.869284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.190447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.512496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.821720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.166202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.486039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.796413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.117555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.440394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.751578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.072699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.387565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.740059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.052202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.380060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.698060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.036297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.362511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.698068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.040895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.399244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.739190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.073725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.409477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.782715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.070717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.370184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.653622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.958745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.255749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.543993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.836305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.136836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.422924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.721435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.006306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.306819] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2680.310125] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2680.311453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.613603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.911538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.185600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.485638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.761535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.033185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.318028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.628416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.929734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.227083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.518103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.834829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.123720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.403967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.670611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.962015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.247255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.531758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.798610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.086200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.384343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.667760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.963119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.277533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.574501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.874659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.181465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.505049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.801050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.101694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.401946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.719996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.026505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.347646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.666038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.987689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.305411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.632569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.946589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.275357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.595589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.891348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.184573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.505632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.811457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.134100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.443328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.800911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.107414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.419090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.728801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.062474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.337325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.639605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.920679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.227746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.501339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.792393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.083760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.399909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.700742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.008943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.304979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.622789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.926038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.233731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.521151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.840137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.132588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.436133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.723152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.015628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.324116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.633296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.945492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.289309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.627416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.958669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.264739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.589993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.898666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.217790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.521371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.831228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.102506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.387236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.689465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.004630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.306740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.599728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.876665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.176210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.473504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.789919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.085710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.405076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.705366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.019420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.323714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.635976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.932060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.223113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.509220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.850958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.147974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.448776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.747791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.077997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.381731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.680376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.974335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.279549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.567154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.871234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.169130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.492790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.784598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.084807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.364828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.662579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.941330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.222979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.512610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.818190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.098038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.389572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.668873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.962580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.247665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.525033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.813746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.102687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.397356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.673189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.962937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.252090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.532759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.831963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.126255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.422588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.721236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.014869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.307656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.619799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.928087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.254260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.541948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.852706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.133828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.421473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.711464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.012656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.312493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.597679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.865727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.154968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.441232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.724485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.994582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.300645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.589786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.906695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.205573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.524293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.833695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.141805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.445896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.773456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.071562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.370128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.659233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.976194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.278840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.603294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.908531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.238459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.537653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.867764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.182825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.502304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.809366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.120183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.404154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.742699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.042431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.367545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.673866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.004529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.322206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.623637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.927222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.240431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.529561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.850415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.168689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.508951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.818633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.131902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.426037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.734230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.066035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.346027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.644942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.965105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.281217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.587302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.897172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.235140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.551994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.851346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.158831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.473934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.782722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.082499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.366205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.669543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.967899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.286917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.585080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.882557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.185858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.482448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.779955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.120528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.432315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.727228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.022587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.345850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.642744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.956931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.253067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.582470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.915881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.225818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.536463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.849907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.132258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.446870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.757573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.079174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.379437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.680071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.984143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.319899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.632300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.956221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.283143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.621369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.939754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.250813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.571665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.897411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.203821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.512466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.822674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.127266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.431253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.757953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.075154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.406244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.737540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.099113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.445026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.736222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.007036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.278089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.581671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.885017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.174778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.490526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.811435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.126681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.432237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.725691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.052862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.348647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.671884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.996049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.322274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.628560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.923262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.227295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.551031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.848718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.153437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.438040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.747307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.070812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.376575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.683884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.005876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.316317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.629548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.920062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.232847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.523161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.822288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.143594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.460861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.769285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.085147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.389199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.717254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.022916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.319824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.633432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.945334] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2773.948580] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 2773.949935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.950747] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2773.956957] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) Skipped 1 previous similar message [ 2774.277124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.591860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.890342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.216767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.537532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.850279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.177813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.504271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.810275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.114466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.422165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.728070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.037301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.338389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.649821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.957074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.258874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.553717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.857826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.176536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.477908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.788379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.098100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.410611] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2781.414544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.417602] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 2781.420030] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2781.422815] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) Skipped 1 previous similar message [ 2781.703161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.039298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.343318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.664729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.960212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.269494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.562224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.870028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.159478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.468922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.758849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.077283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.384350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.677371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.982799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.290221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.590022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.886805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.186100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.505923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.797499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.089351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.377466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.661048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.941684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.240033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.554220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.860560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.160541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.484655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.829895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.187425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.525538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.859531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.185739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.483808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.770855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.030907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.296164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.577064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.859632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.138275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.417771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.692924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.976052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.255729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.539513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.831328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.117042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.401367] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2796.404104] Lustre: Skipped 418 previous similar messages [ 2796.410913] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2796.415740] Lustre: Skipped 418 previous similar messages [ 2796.417768] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2796.419557] Lustre: Skipped 418 previous similar messages [ 2796.420814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.423900] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2796.425666] Lustre: Skipped 418 previous similar messages [ 2796.704134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.999185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.294577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.580291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.852532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.130396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.412774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.702270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.971677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.246955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.527187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.798341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.067198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.338845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.617346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.889764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.179327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.462368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.747541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.027063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.310251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.596565] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2802.600545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.600948] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 2802.609939] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2802.614676] LustreError: 162270:0:(osp_precreate.c:1356:osp_precreate_thread()) Skipped 1 previous similar message [ 2802.896660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.174497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.458336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.756682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.048456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.323472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.603995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.894617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.166491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.450814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.749502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.059536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.335542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.607076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.888877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.197166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.485442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.769301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.044715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.330440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.599443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.889681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.171634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.461672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.748743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.038348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.326870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.639480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.944694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.241192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.556396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.879695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.183796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.480216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.788938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.091657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.370215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.649380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.925825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.219891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.504495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.804271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.099846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.409369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.701195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.992480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.292357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.589907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.877976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.171233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.455616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.745185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.027274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.319774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.613787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.923774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.224092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.511212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.819249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.129155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.438536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.717867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.012725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.352192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.651447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.935705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.241023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.547060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.826225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.125685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.425573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.768263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.089761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.421335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.764237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.130728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.448285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.766261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.051925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.364444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.660961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.940123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.207445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.485752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.784888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.073222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.356941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.643832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.938529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.227835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.515925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.806815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.075961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.346684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.628166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.906605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.186187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.469662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.743416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.027098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.300045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.585175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.854504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.130942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.414867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.699826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.983301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.288400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.588407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.879641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.148673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.430395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.718844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.015693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.321086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.616988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.914562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.214482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.513850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.819484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.123084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.416019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.718442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.011305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.299283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.599557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.884894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.170349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.447217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.749915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.032718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.323888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.611826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.906360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.276074] LustreError: 163842:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2903.279824] LustreError: 162270:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2966.496369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2966.499421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2967.522229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2967.523771] Lustre: Skipped 1 previous similar message [ 2969.712812] Lustre: server umount lustre-OST0000 complete [ 2971.615892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2971.618433] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2977.051612] Lustre: server umount lustre-MDT0000 complete [ 2977.759937] LustreError: 194157:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2977.766093] LustreError: 194157:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2978.342070] LustreError: 162229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088884 with bad export cookie 12881135789426918321 [ 2978.344529] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2978.347705] LustreError: 162229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2978.476315] Lustre: server umount lustre-MDT0001 complete [ 2981.817430] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 2982.852309] Key type lgssc unregistered [ 2982.992442] LNet: 216933:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2982.995347] LNetError: 216933:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2983.005451] LNet: Removed LNI 192.168.201.160@tcp [ 2983.293390] Key type .llcrypt unregistered [ 2983.294265] Key type ._llcrypt unregistered [ 2991.999608] Key type ._llcrypt registered [ 2992.000539] Key type .llcrypt registered [ 2992.034174] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 2992.456462] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2992.480656] alg: No test for adler32 (adler32-zlib) [ 2993.333031] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 2993.411452] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 2994.991140] Key type lgssc registered [ 2995.313353] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2997.739725] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2997.749834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2998.922806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3000.170909] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3002.894617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3003.014576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3004.249508] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3005.967227] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3007.118356] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3009.344265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3009.451341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3010.471594] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3011.585883] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3013.262712] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3016.078401] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3016.678606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3016.682870] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3016.685178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.689436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3016.689454] LustreError: 218326:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3017.696334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3017.697086] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3017.697724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3017.705867] Lustre: Skipped 1 previous similar message [ 3019.412367] Lustre: server umount lustre-OST0000 complete [ 3036.767094] LustreError: 218326:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3047.286947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3047.390830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3049.203077] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3050.864681] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3057.638875] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.642621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.642673] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3057.647772] Lustre: Skipped 1 previous similar message [ 3062.724531] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 11 sec [ 3063.841626] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ff558cc9000.ost_server_uuid 50 [ 3064.276581] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ff558cc9000.ost_server_uuid in FULL state after 0 sec [ 3067.874152] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3067.874302] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3067.877713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3067.880114] Lustre: Skipped 1 previous similar message [ 3067.881907] Lustre: Skipped 1 previous similar message [ 3071.309319] Lustre: server umount lustre-OST0000 complete [ 3072.992315] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3072.993152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3072.995819] Lustre: Skipped 1 previous similar message [ 3072.998309] Lustre: Skipped 1 previous similar message [ 3078.112629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3078.114287] Lustre: Skipped 2 previous similar messages [ 3078.677062] Lustre: server umount lustre-MDT0000 complete [ 3079.921424] LustreError: 218286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780088986 with bad export cookie 16835178237552187536 [ 3079.924628] LustreError: 218286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3079.925900] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3080.054066] Lustre: server umount lustre-MDT0001 complete [ 3083.793292] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 3084.850649] Key type lgssc unregistered [ 3084.980303] LNet: 222960:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3084.982545] LNetError: 222960:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3084.995247] LNet: Removed LNI 192.168.201.160@tcp [ 3085.313148] Key type .llcrypt unregistered [ 3085.314935] Key type ._llcrypt unregistered [ 3096.742766] Key type ._llcrypt registered [ 3096.743747] Key type .llcrypt registered [ 3096.790703] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3097.252697] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3097.262902] alg: No test for adler32 (adler32-zlib) [ 3098.125232] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 3098.212594] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 3099.807211] Key type lgssc registered [ 3100.206193] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3103.234304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3103.240306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3104.402482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3105.726109] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3108.645674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3108.766849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3110.057435] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3111.078623] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3113.303266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3113.416554] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3114.470091] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3115.298470] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3118.149398] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3118.214579] Lustre: lustre-OST0001: new disk, initializing [ 3118.216344] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3118.218740] Lustre: 226903:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3118.254773] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3120.149880] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3121.930000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3121.933133] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3121.971176] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3125.592665] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3131.158692] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3139.551827] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3139.554127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3139.558278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3142.625675] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3142.627132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3142.630281] Lustre: Skipped 2 previous similar messages [ 3142.632509] Lustre: Skipped 2 previous similar messages [ 3144.607437] Lustre: server umount lustre-MDT0000 complete [ 3145.811143] LustreError: 225745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089052 with bad export cookie 4922440622954102420 [ 3145.814898] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3145.817104] LustreError: 225745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3145.921226] Lustre: server umount lustre-MDT0001 complete [ 3156.929098] Lustre: server umount lustre-OST0000 complete [ 3168.412754] Lustre: server umount lustre-OST0001 complete [ 3175.450806] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3178.950077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3179.139265] LustreError: 229975:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3179.149158] LustreError: 229975:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3179.167635] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3180.584558] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3183.547483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3185.134918] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3186.224916] Lustre: 231083:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3188.553124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3188.690470] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3188.692994] Lustre: Skipped 1 previous similar message [ 3189.734155] LustreError: 231438:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3189.740400] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 3190.696762] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3193.607241] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3195.564354] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3198.586858] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3199.704373] Lustre: 232915:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3204.064146] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3204.066174] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3204.070437] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3204.072448] Lustre: Skipped 1 previous similar message [ 3208.979507] Lustre: server umount mylustre-MDT0000 complete [ 3210.084072] LustreError: 229950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089116 with bad export cookie 4922440622954105185 [ 3210.086833] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3210.087918] LustreError: 229950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3210.184475] Lustre: server umount mylustre-MDT0001 complete [ 3220.549583] Lustre: server umount mylustre-OST0000 complete [ 3232.094205] Lustre: server umount mylustre-OST0001 complete [ 3238.984509] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3242.555243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3242.706741] LustreError: 235202:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3242.731479] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3242.733552] Lustre: Skipped 1 previous similar message [ 3244.021826] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3246.878969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3248.350390] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3249.331178] Lustre: 236309:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3251.498163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3252.643983] LustreError: 236662:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3252.647707] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 3253.609514] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3256.417060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3259.525442] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3264.891338] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3267.714882] Lustre: 238145:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3272.672625] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3272.677914] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3272.687925] Lustre: Skipped 3 previous similar messages [ 3272.691147] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3272.694258] Lustre: Skipped 4 previous similar messages [ 3277.840172] Lustre: server umount tfs-MDT0000 complete [ 3278.306618] LustreError: 236686:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3278.325065] LustreError: 236686:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3279.983236] LustreError: 235179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089186 with bad export cookie 4922440622954106851 [ 3279.984428] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3279.989875] LustreError: 235179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3280.419349] Lustre: server umount tfs-MDT0001 complete [ 3295.162273] Lustre: server umount tfs-OST0000 complete [ 3310.487746] Lustre: server umount tfs-OST0001 complete [ 3330.331962] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3337.377170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3337.612484] LustreError: 240378:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3337.656656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3337.659634] Lustre: Skipped 3 previous similar messages [ 3339.717292] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3344.878243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3347.038945] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3348.445414] Lustre: 241484:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3351.880823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3355.111636] LustreError: 241835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3355.125983] LustreError: 241835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3355.233308] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3360.053060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3363.097217] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3363.245187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 3367.712784] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3369.827977] Lustre: 243321:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3373.901373] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:14:39 (1780089279) [ 3388.899645] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3388.905732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3388.910028] Lustre: Skipped 5 previous similar messages [ 3388.916783] Lustre: Skipped 7 previous similar messages [ 3393.147350] Lustre: server umount lustre-MDT0000 complete [ 3394.021166] LustreError: 241109:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3394.035632] LustreError: 241109:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3395.174200] LustreError: 240354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089301 with bad export cookie 4922440622954108293 [ 3395.179168] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3395.185470] LustreError: 240354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3399.146831] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3399.152019] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3399.155473] Lustre: Skipped 2 previous similar messages [ 3401.427743] Lustre: server umount lustre-MDT0001 complete [ 3409.615307] Lustre: server umount lustre-OST0000 complete [ 3417.776331] Lustre: server umount lustre-OST0001 complete [ 3421.145617] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 3425.883862] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3446.647292] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3446.647292] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3446.647292] [ 3446.658137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3446.802474] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3446.827217] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3446.889894] Lustre: lustre-MDT0000: new disk, initializing [ 3446.942049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.945269] Lustre: Skipped 3 previous similar messages [ 3446.953609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3449.304202] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3456.203073] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3456.203073] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3456.203073] [ 3456.225993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3456.295744] Lustre: 246464:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3456.330240] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3456.333217] Lustre: Skipped 1 previous similar message [ 3456.370370] Lustre: lustre-MDT0001: new disk, initializing [ 3456.435557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3456.440501] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3458.744187] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3464.767492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3464.916674] Lustre: lustre-OST0000: new disk, initializing [ 3464.919379] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3464.925102] Lustre: 248013:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3466.184798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3466.195768] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3466.240276] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3468.702263] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3475.145043] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3475.221209] Lustre: lustre-OST0001: new disk, initializing [ 3475.223127] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3475.228176] Lustre: 248870:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3476.846707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3476.851434] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3476.920367] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3478.127575] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3484.371907] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:16:30 (1780089390) [ 3490.688932] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:16:36 (1780089396) [ 3507.172610] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3507.172941] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3507.178371] LustreError: Skipped 1 previous similar message [ 3507.189316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3507.193571] Lustre: Skipped 4 previous similar messages [ 3508.688073] Lustre: server umount lustre-OST0000 complete [ 3512.803092] LustreError: 248151:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3512.803443] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3512.810736] LustreError: 248151:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3517.029086] Lustre: server umount lustre-MDT0000 complete [ 3518.959974] LustreError: 246457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089425 with bad export cookie 4922440622954109623 [ 3518.968115] LustreError: 246457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3518.971199] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3533.279107] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3533.390401] Lustre: server umount lustre-MDT0001 complete [ 3538.236739] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 3549.761036] Lustre: server umount lustre-OST0001 complete [ 3552.927126] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 3562.003376] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3581.196747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3581.328585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3581.356296] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3581.400729] Lustre: lustre-MDT0000: new disk, initializing [ 3581.460944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3581.464530] Lustre: Skipped 3 previous similar messages [ 3581.474173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3583.505725] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3589.880042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3589.970261] Lustre: 253329:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3590.116403] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3592.060825] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3596.330416] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3598.520184] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3602.142376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3602.293189] Lustre: lustre-OST0000: new disk, initializing [ 3602.296870] Lustre: Skipped 1 previous similar message [ 3602.301727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3602.304463] Lustre: Skipped 2 previous similar messages [ 3602.307934] Lustre: 255070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3604.258133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3604.262824] Lustre: Skipped 1 previous similar message [ 3604.267835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3604.339936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3605.223967] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3609.207990] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3614.690238] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3614.696104] Lustre: Skipped 5 previous similar messages [ 3614.698365] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3614.703139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3614.705448] Lustre: Skipped 7 previous similar messages [ 3618.746166] Lustre: server umount lustre-OST0000 complete [ 3635.167188] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3635.242537] Lustre: server umount lustre-MDT0000 complete [ 3636.911752] LustreError: 253320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089543 with bad export cookie 4922440622954111093 [ 3636.917271] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3636.919609] LustreError: 253320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3637.097948] Lustre: server umount lustre-MDT0001 complete [ 3643.148592] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 3644.562047] Key type lgssc unregistered [ 3644.707447] LNet: 256700:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3644.711170] LNetError: 256700:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3644.719452] LNet: Removed LNI 192.168.201.160@tcp [ 3645.161161] Key type .llcrypt unregistered [ 3645.162852] Key type ._llcrypt unregistered [ 3660.191701] Key type ._llcrypt registered [ 3660.193957] Key type .llcrypt registered [ 3660.286365] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3660.813549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3660.878091] alg: No test for adler32 (adler32-zlib) [ 3661.805563] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 3661.942458] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 3663.559145] Key type lgssc registered [ 3664.175183] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3668.094650] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3668.108164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3669.432947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3671.688293] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3676.442632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3676.640783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3678.866964] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3681.609596] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3683.559730] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3687.072865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3687.235550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3689.870194] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3692.391380] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3692.917584] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3693.869954] Lustre: server umount lustre-OST0000 complete [ 3697.121259] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3697.127989] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3697.139313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3700.197814] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3700.197933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3700.225510] Lustre: Skipped 1 previous similar message [ 3701.804269] Lustre: server umount lustre-MDT0000 complete [ 3702.320759] LustreError: 258124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.332940] LustreError: 258124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3704.065407] LustreError: 258104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089610 with bad export cookie 5975870061528357739 [ 3704.073380] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3704.299303] Lustre: server umount lustre-MDT0001 complete [ 3709.970637] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 3711.338661] Key type lgssc unregistered [ 3711.478348] LNet: 261345:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3711.481146] LNetError: 261345:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3711.489366] LNet: Removed LNI 192.168.201.160@tcp [ 3711.861368] Key type .llcrypt unregistered [ 3711.862858] Key type ._llcrypt unregistered [ 3720.930066] Key type ._llcrypt registered [ 3720.931288] Key type .llcrypt registered [ 3720.975516] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3721.427424] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3721.440077] alg: No test for adler32 (adler32-zlib) [ 3722.319850] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 3722.429376] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 3724.031248] Key type lgssc registered [ 3724.585255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3728.884263] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3728.899176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3744.415431] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3749.535389] LustreError: 262499:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.160@tcp: failed processing log, type 4: rc = -110 [ 3776.159237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3778.327546] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3780.525752] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3781.354979] Lustre: server umount lustre-OST0000 complete [ 3788.063930] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 3789.291606] Key type lgssc unregistered [ 3789.422342] LNet: 263997:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3789.424761] LNetError: 263997:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3789.435583] LNet: Removed LNI 192.168.201.160@tcp [ 3789.794559] Key type .llcrypt unregistered [ 3789.795973] Key type ._llcrypt unregistered [ 3808.871306] Key type ._llcrypt registered [ 3808.873110] Key type .llcrypt registered [ 3808.931612] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3809.400420] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3809.410790] alg: No test for adler32 (adler32-zlib) [ 3810.287370] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 3810.376805] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 3811.975194] Key type lgssc registered [ 3812.458183] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3817.994933] loop0: detected capacity change from 0 to 204800000 [ 3818.078293] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3818.085393] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3818.226108] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3818.624109] loop0: detected capacity change from 0 to 204800000 [ 3818.737570] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3818.742814] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3818.886164] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3819.275848] loop0: detected capacity change from 0 to 204800000 [ 3819.341648] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3819.345969] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3819.398663] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3819.852911] loop0: detected capacity change from 0 to 204800000 [ 3819.918197] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3820.420225] loop0: detected capacity change from 0 to 204800000 [ 3820.436357] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3822.334306] loop0: detected capacity change from 0 to 204800000 [ 3822.349066] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3824.138765] loop0: detected capacity change from 0 to 204800000 [ 3824.150917] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3825.683813] loop0: detected capacity change from 0 to 204800000 [ 3825.697510] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3827.568384] loop0: detected capacity change from 0 to 204800000 [ 3827.602540] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3827.616668] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3827.654367] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3827.660209] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3827.665348] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3827.671253] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3827.680073] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3827.684451] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3827.688531] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3827.692100] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3827.696641] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3827.701308] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3827.706033] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3827.714112] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3827.719364] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3827.724156] Lustre: 267369:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3830.683326] Lustre: server umount MGS complete [ 3831.068871] loop0: detected capacity change from 0 to 204800000 [ 3831.096771] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3831.294844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3831.315079] Lustre: lustre-MDT0000: Aborting client recovery [ 3831.671685] loop1: detected capacity change from 0 to 204800000 [ 3831.741155] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3831.770782] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3831.775317] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3831.779967] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3831.785414] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3831.790091] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3831.795353] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3831.801091] Lustre: 267799:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3831.811593] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3831.905433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3831.934366] Lustre: lustre-MDT0001: Aborting client recovery [ 3832.324349] loop2: detected capacity change from 0 to 204800000 [ 3832.364313] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3832.409107] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3832.415456] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3832.420715] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3832.426578] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3832.432454] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3832.438432] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3832.443942] Lustre: 267882:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3832.499308] Lustre: Found index 0 for lustre-OST0000, updating log [ 3832.514629] Lustre: 267900:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3832.552328] Lustre: lustre-OST0000: Aborting recovery [ 3832.912369] loop3: detected capacity change from 0 to 204800000 [ 3832.944974] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3832.976667] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3832.982708] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3832.988368] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3832.993691] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3832.999597] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3833.005861] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3833.011539] Lustre: 267966:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3833.033942] Lustre: 267971:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3833.061544] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3833.063752] Lustre: Skipped 1 previous similar message [ 3833.071551] Lustre: lustre-OST0001: Aborting recovery [ 3838.444048] LustreError: 267743:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 3840.485099] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3840.493855] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3840.498171] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3840.507104] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3841.506387] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3841.506611] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3844.816530] LustreError: 267709:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 242us: rc = -2 [ 3856.865117] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3856.865453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3856.869811] Lustre: Skipped 3 previous similar messages [ 3856.872518] Lustre: Skipped 2 previous similar messages [ 3861.987350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3861.989609] Lustre: Skipped 4 previous similar messages [ 3862.245446] Lustre: server umount lustre-MDT0000 complete [ 3862.588719] LustreError: 267807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089769 with bad export cookie 4483763953149410635 [ 3862.593405] LustreError: 267807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3862.593808] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3867.103814] LustreError: 267908:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3867.104030] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3867.104695] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3867.110715] LustreError: 267908:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3867.115606] Lustre: Skipped 1 previous similar message [ 3868.811804] Lustre: server umount lustre-MDT0001 complete [ 3875.357225] Lustre: server umount lustre-OST0000 complete [ 3890.143124] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3890.219101] Lustre: server umount lustre-OST0001 complete [ 3893.754196] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:23:19 (1780089799) [ 3905.269213] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 3909.937748] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 3927.225741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.347035] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3927.371807] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3927.414414] Lustre: lustre-MDT0000: new disk, initializing [ 3927.451134] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3927.460175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3929.223574] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3934.455541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3934.503906] Lustre: 272001:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3934.541338] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3934.543983] Lustre: Skipped 1 previous similar message [ 3934.583676] Lustre: lustre-MDT0001: new disk, initializing [ 3934.624811] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3934.642599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3934.647185] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3936.487353] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3940.424913] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3941.998576] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3944.905727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3945.025909] Lustre: lustre-OST0000: new disk, initializing [ 3945.028382] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3945.032761] Lustre: 273742:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3945.077268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3947.631672] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 3951.301384] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3952.129255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3952.133215] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3952.178206] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3959.499996] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3962.391449] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3969.952045] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3976.648209] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4018.657580] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4018.657678] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4018.665174] LustreError: Skipped 1 previous similar message [ 4018.670857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4018.680757] Lustre: Skipped 2 previous similar messages [ 4019.763517] Lustre: server umount lustre-OST0000 complete [ 4021.728089] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4021.731242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.736101] Lustre: Skipped 1 previous similar message [ 4023.777067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4023.779832] Lustre: Skipped 2 previous similar messages [ 4027.703190] Lustre: server umount lustre-MDT0000 complete [ 4028.896223] LustreError: 272794:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4028.904645] LustreError: 272794:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4029.466355] LustreError: 271994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780089936 with bad export cookie 4483763953149413344 [ 4029.468562] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4029.474536] LustreError: 271994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4029.622587] Lustre: server umount lustre-MDT0001 complete [ 4033.749947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4035.446172] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4037.673228] Lustre: server umount MGS complete [ 4042.372336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4042.571460] LustreError: 277474:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4042.582761] LustreError: 277487:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4042.615512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4044.046427] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4047.639336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4049.319314] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4051.343912] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4052.745276] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4055.505245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4058.014258] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4060.348406] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4093.919985] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4093.922971] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4093.925136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4093.927182] Lustre: Skipped 2 previous similar messages [ 4100.052766] Lustre: server umount lustre-OST0000 complete [ 4104.160834] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4104.160863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4104.165912] Lustre: Skipped 1 previous similar message [ 4107.609883] Lustre: server umount lustre-MDT0000 complete [ 4109.118518] LustreError: 277454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780090015 with bad export cookie 4483763953149415185 [ 4109.122418] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4109.122849] LustreError: 277454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4109.250581] Lustre: server umount lustre-MDT0001 complete [ 4113.409762] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 4114.491582] Key type lgssc unregistered [ 4114.628399] LNet: 281081:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4114.631905] LNetError: 281081:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4114.638338] LNet: Removed LNI 192.168.201.160@tcp [ 4114.963190] Key type .llcrypt unregistered [ 4114.964599] Key type ._llcrypt unregistered [ 4128.679340] Key type ._llcrypt registered [ 4128.680888] Key type .llcrypt registered [ 4128.727280] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 4135.887469] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 4136.367198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4136.374506] alg: No test for adler32 (adler32-zlib) [ 4137.246971] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 4137.355659] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 4138.943826] Key type lgssc registered [ 4139.376431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4152.644652] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4152.650334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4153.756535] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4153.774840] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4153.813338] Lustre: lustre-MDT0000: new disk, initializing [ 4153.853629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4153.862852] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4155.447957] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4160.443620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4160.495180] Lustre: 284975:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4160.523262] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4160.525487] Lustre: Skipped 1 previous similar message [ 4160.564818] Lustre: lustre-MDT0001: new disk, initializing [ 4160.597584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4160.612204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4160.616630] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4162.301446] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4167.406130] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4170.817919] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4179.473847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4179.771590] Lustre: lustre-OST0000: new disk, initializing [ 4179.785953] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4179.820243] Lustre: 286718:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4179.956681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4185.688753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4185.699096] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4185.763757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4187.495975] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4196.026817] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4203.101470] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4215.188384] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4222.205206] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4226.252305] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4268.001756] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4268.002172] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4268.014073] Lustre: Skipped 1 previous similar message [ 4268.014257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4271.583996] Lustre: server umount lustre-OST0000 complete [ 4278.240437] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4278.240957] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4278.254707] Lustre: Skipped 1 previous similar message [ 4278.255699] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4278.264898] Lustre: Skipped 2 previous similar messages [ 4280.887617] Lustre: server umount lustre-MDT0000 complete [ 4284.101952] LustreError: 284968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780090190 with bad export cookie 4914055369522437812 [ 4284.102854] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4284.110243] LustreError: 284968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4284.336504] Lustre: server umount lustre-MDT0001 complete [ 4290.662315] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4294.131605] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4297.880452] Lustre: server umount MGS complete [ 4306.636993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4306.904436] LustreError: 290450:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4306.913962] LustreError: 290450:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4306.924611] LustreError: 290463:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4306.971441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4310.334290] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4312.033605] LustreError: 290451:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4316.880877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4317.278845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4320.691465] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4325.015128] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4328.039911] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4332.801670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4333.041293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4338.140367] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4342.154617] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4382.177391] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4382.186678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4382.192446] Lustre: Skipped 1 previous similar message [ 4384.802573] Lustre: server umount lustre-OST0000 complete [ 4388.833703] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4388.844420] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4388.859102] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4388.865357] Lustre: Skipped 1 previous similar message [ 4394.457844] Lustre: server umount lustre-MDT0000 complete [ 4397.348787] LustreError: 292856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780090303 with bad export cookie 4914055369522439625 [ 4397.350302] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4397.366922] LustreError: 292856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4397.603924] Lustre: server umount lustre-MDT0001 complete [ 4408.024774] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 4410.264042] Key type lgssc unregistered [ 4410.571740] LNet: 294056:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4410.580252] LNetError: 294056:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4410.602984] LNet: Removed LNI 192.168.201.160@tcp [ 4411.241466] Key type .llcrypt unregistered [ 4411.243457] Key type ._llcrypt unregistered [ 4430.122860] Key type ._llcrypt registered [ 4430.127859] Key type .llcrypt registered [ 4430.292084] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 4444.685782] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 4445.836718] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4445.870100] alg: No test for adler32 (adler32-zlib) [ 4446.835612] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 4446.999980] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 4448.623160] Key type lgssc registered [ 4449.481743] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4469.079794] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 4481.797604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4481.825836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4483.075484] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4483.121951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4483.270527] Lustre: lustre-MDT0000: new disk, initializing [ 4483.374221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4483.405512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4487.069601] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4496.997565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4497.274526] Lustre: lustre-OST0000: new disk, initializing [ 4497.278146] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4497.282224] Lustre: Skipped 1 previous similar message [ 4497.286578] Lustre: 298037:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4497.357096] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4503.604526] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 4504.202833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4504.216311] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4504.257316] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5200.354966] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5200.364511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5200.729762] Lustre: server umount lustre-MDT0000 complete [ 5203.042462] LustreError: 297243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780091109 with bad export cookie 13575493715658380836 [ 5203.052913] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5205.196864] Lustre: server umount lustre-OST0000 complete [ 5222.291520] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 5230.272480] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 5283.076834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5283.557832] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5283.599394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5283.777525] Lustre: lustre-MDT0000: new disk, initializing [ 5283.908841] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5283.927409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5289.367458] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5306.737132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5307.002214] Lustre: 302303:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5307.133928] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5307.144135] Lustre: Skipped 1 previous similar message [ 5307.331484] Lustre: lustre-MDT0001: new disk, initializing [ 5307.462346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5307.500097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5307.529959] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5314.903931] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5327.102851] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5330.970084] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5338.011347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5338.320907] Lustre: lustre-OST0000: new disk, initializing [ 5338.324611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5338.331398] Lustre: 304056:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5338.445691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5345.403698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5345.409738] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5345.587975] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5346.360561] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5354.349588] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5359.478876] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5359.760628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5365.617331] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5366.024718] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5371.362087] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5371.369586] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5371.382245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5374.961880] Lustre: server umount lustre-OST0000 complete [ 5379.043806] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5379.055792] LustreError: Skipped 1 previous similar message [ 5379.061271] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5379.083509] Lustre: Skipped 1 previous similar message [ 5379.092757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5379.101449] Lustre: Skipped 1 previous similar message [ 5381.609207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5381.616163] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5386.721744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5386.731562] Lustre: Skipped 2 previous similar messages [ 5393.375163] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5393.639689] Lustre: server umount lustre-MDT0000 complete [ 5397.503885] LustreError: 302295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780091304 with bad export cookie 13575493715661321655 [ 5397.507942] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5397.511464] LustreError: 302295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5397.957637] Lustre: server umount lustre-MDT0001 complete [ 5401.619821] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5403.620500] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:48:28 (1780091308) [ 5410.223046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5410.611813] LustreError: 306227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5410.627932] LustreError: 306227:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5410.680180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5414.942726] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5415.905660] LustreError: 306228:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5421.025852] LustreError: 306227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5423.166354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5423.574480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5428.201731] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5433.731844] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5437.883914] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5445.320911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5445.589559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5451.384546] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5456.528093] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5463.265757] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5463.464182] Lustre: lustre-OST0001: new disk, initializing [ 5463.468316] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5463.477627] Lustre: 308720:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5463.583690] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5469.180438] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5470.272117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5470.285621] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5470.335157] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5471.394873] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5471.401413] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5472.420699] LustreError: 309137:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5472.437922] LustreError: 309137:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5474.466811] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5474.485639] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5478.633243] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5478.647249] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5478.802693] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5478.961393] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5482.764574] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ff544e33800.ost_server_uuid 50 [ 5484.314162] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ff544e33800.ost_server_uuid in FULL state after 0 sec [ 5486.947130] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5486.957804] LustreError: 309131:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [ 5487.665263] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ff544e33800.ost_server_uuid 50 [ 5489.092609] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9ff544e33800.ost_server_uuid in FULL state after 0 sec [ 5491.206661] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5505.274528] Lustre: lustre-OST0001: Not available for connect from 192.168.201.60@tcp (stopping) [ 5505.281102] Lustre: Skipped 2 previous similar messages [ 5506.542307] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5506.554889] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5506.559232] LustreError: Skipped 2 previous similar messages [ 5506.578699] Lustre: Skipped 1 previous similar message [ 5510.327371] Lustre: server umount lustre-OST0001 complete [ 5510.400085] LustreError: 307823:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5516.771669] LustreError: 309005:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5516.774178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5516.787575] LustreError: 309005:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5518.186433] Lustre: server umount lustre-OST0000 complete [ 5521.893777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5521.906280] Lustre: Skipped 5 previous similar messages [ 5527.681730] Lustre: server umount lustre-MDT0000 complete [ 5531.271410] LustreError: 306209:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780091437 with bad export cookie 13575493715661322537 [ 5531.292756] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5531.712091] Lustre: server umount lustre-MDT0001 complete [ 5542.371757] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 5544.554116] Key type lgssc unregistered [ 5544.847745] LNet: 311166:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5544.858815] LNetError: 311166:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5545.901147] LNet: Removed LNI 192.168.201.160@tcp [ 5546.845496] Key type .llcrypt unregistered [ 5546.848842] Key type ._llcrypt unregistered [ 5573.645570] Key type ._llcrypt registered [ 5573.647531] Key type .llcrypt registered [ 5573.737723] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 5574.584574] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5574.672135] alg: No test for adler32 (adler32-zlib) [ 5575.754871] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 5575.922610] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 5577.599490] Key type lgssc registered [ 5578.563597] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5585.201863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5585.217625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5586.599872] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5590.725722] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5596.354063] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5597.154028] LustreError: 312572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5597.168469] LustreError: 312572:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5602.285378] LustreError: 312573:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5603.330753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5603.754181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5608.298269] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5617.869310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5618.242995] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5625.094044] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5628.514094] LustreError: 314123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5630.387069] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5631.607932] LustreError: 314121:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5631.638442] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5631.655060] LustreError: 314121:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5637.101175] LustreError: 314702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5637.117666] LustreError: 314702:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5637.735292] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5637.968831] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5643.266488] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5644.884270] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5650.994812] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5658.718523] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5658.972457] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5760.995306] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5761.029797] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5761.044913] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5766.115370] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5766.125084] Lustre: Skipped 2 previous similar messages [ 5771.241692] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5771.246959] Lustre: Skipped 1 previous similar message [ 5775.332758] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5775.433187] Lustre: server umount lustre-OST0001 complete [ 5776.379550] LustreError: 314702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5776.405768] LustreError: 314702:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5782.498385] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5782.508249] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5782.521951] Lustre: Skipped 1 previous similar message [ 5782.531401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5785.571438] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5785.575920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5786.599688] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5786.607547] Lustre: Skipped 1 previous similar message [ 5796.319506] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5796.533154] Lustre: server umount lustre-OST0000 complete [ 5801.952620] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5801.971569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5801.976551] Lustre: Skipped 1 previous similar message [ 5801.979338] Lustre: Skipped 2 previous similar messages [ 5806.947649] Lustre: server umount lustre-MDT0000 complete [ 5807.072340] LustreError: 315489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5807.093626] LustreError: 315489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 5810.345848] LustreError: 312554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780091716 with bad export cookie 5784125793562647199 [ 5810.347138] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5810.356627] LustreError: 312554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5810.695821] Lustre: server umount lustre-MDT0001 complete [ 5820.473730] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 5822.994800] Key type lgssc unregistered [ 5823.257820] LNet: 317125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5823.266826] LNetError: 317125:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5823.285163] LNet: Removed LNI 192.168.201.160@tcp [ 5823.941103] Key type .llcrypt unregistered [ 5823.942963] Key type ._llcrypt unregistered [ 5868.700973] Key type ._llcrypt registered [ 5868.704676] Key type .llcrypt registered [ 5868.846085] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 5870.330471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5870.359887] alg: No test for adler32 (adler32-zlib) [ 5871.558158] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 5871.777955] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 5873.505818] Key type lgssc registered [ 5874.898808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5884.453478] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5884.469235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5885.975375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5890.884413] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5896.162884] LustreError: 319401:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5896.176106] LustreError: 319401:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5899.591892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5900.047708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5904.617673] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5907.985327] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5916.434528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5916.814369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5922.920287] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5924.995843] LustreError: 320858:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5930.467587] LustreError: 321315:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5931.519052] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5931.820470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5932.052384] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5937.139698] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5938.193550] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 5948.412775] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5953.963245] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5966.816849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5966.821311] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5966.830037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5967.842799] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5967.847404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5967.867839] Lustre: Skipped 2 previous similar messages [ 5967.882192] Lustre: Skipped 3 previous similar messages [ 5971.947143] Lustre: server umount lustre-MDT0000 complete [ 5972.963715] LustreError: 322151:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5972.973158] LustreError: 322151:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5975.691153] LustreError: 319381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780091882 with bad export cookie 7771101246386329892 [ 5975.699060] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5975.708043] LustreError: 319381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5976.119033] Lustre: server umount lustre-MDT0001 complete [ 5990.326516] Lustre: server umount lustre-OST0000 complete [ 5993.439194] Lustre: 318664:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780091884/real 1780091884] req@ffff8f66be769c00 x1866561516273408/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780091900 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5993.480257] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5994.790952] Lustre: server umount lustre-OST0001 complete [ 6012.490721] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 6023.164797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6023.785031] LustreError: 324580:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6023.854141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6027.808123] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6034.410591] LustreError: 324581:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6034.425655] LustreError: 324581:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6036.189149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6036.475047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6040.772852] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6044.081950] Lustre: 325687:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6051.321292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6057.605758] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6060.848762] LustreError: 326040:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6067.186229] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6067.410194] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6067.413612] Lustre: Skipped 1 previous similar message [ 6068.473796] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6068.485139] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6073.828839] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6081.718497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6085.642387] Lustre: 327526:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6092.770516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6092.788106] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6092.811350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6096.474483] Lustre: server umount lustre-MDT0000 complete [ 6099.426209] LustreError: 324576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6099.465141] LustreError: 324576:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 6100.344070] LustreError: 325306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780092006 with bad export cookie 7771101246386331593 [ 6100.353427] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6100.353614] LustreError: 325306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6100.628886] Lustre: server umount lustre-MDT0001 complete [ 6115.579770] Lustre: server umount lustre-OST0000 complete [ 6129.802462] Lustre: server umount lustre-OST0001 complete [ 6133.347896] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6134.946617] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6136.623733] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:00:41 (1780092041) [ 6158.113344] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 6166.735265] loop0: detected capacity change from 0 to 1099511627776 [ 6167.204669] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6167.209534] *** VALIDATE xfs *** [ 6167.217943] XFS (loop0): Mounting V5 Filesystem [ 6167.246514] XFS (loop0): Ending clean mount [ 6168.364888] loop1: detected capacity change from 0 to 18691697672192 [ 6262.237746] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 6271.404355] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 6305.677853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6305.896871] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6305.919951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6306.003916] Lustre: lustre-MDT0000: new disk, initializing [ 6306.096569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6306.114422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6309.943882] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6321.079259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6321.297632] Lustre: 332621:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6321.366888] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6321.370205] Lustre: Skipped 1 previous similar message [ 6321.447782] Lustre: lustre-MDT0001: new disk, initializing [ 6321.566586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6321.591254] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6326.096796] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6333.427354] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6337.797433] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6344.890302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6345.104604] Lustre: lustre-OST0000: new disk, initializing [ 6345.107363] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6345.112651] Lustre: 334363:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6346.963491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6346.968716] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6347.100230] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6350.566391] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6357.754217] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6361.604810] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6361.799598] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6365.683219] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6365.849486] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6367.713517] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6367.718452] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6367.722518] LustreError: Skipped 1 previous similar message [ 6367.724287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6367.724293] Lustre: Skipped 4 previous similar messages [ 6367.742104] Lustre: Skipped 4 previous similar messages [ 6372.849793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6372.860785] Lustre: Skipped 2 previous similar messages [ 6373.734585] Lustre: server umount lustre-OST0000 complete [ 6377.953088] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6377.963299] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6383.073353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6383.083889] Lustre: Skipped 3 previous similar messages [ 6383.546611] Lustre: server umount lustre-MDT0000 complete [ 6387.550683] LustreError: 332615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780092294 with bad export cookie 7771101246386333140 [ 6387.568176] LustreError: 332615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6387.569933] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6387.990891] Lustre: server umount lustre-MDT0001 complete [ 6397.387955] XFS (loop0): Unmounting Filesystem [ 6402.600992] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:05:07 (1780092307) [ 6409.796936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6410.167981] LustreError: 336773:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6410.259640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6410.273637] Lustre: Skipped 2 previous similar messages [ 6415.318695] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6423.520868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6428.602322] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6433.882477] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6437.441628] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6443.553989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6449.800887] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6456.103915] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6466.529950] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6466.534033] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6466.553722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6466.603609] Lustre: Skipped 2 previous similar messages [ 6471.357968] Lustre: server umount lustre-OST0000 complete [ 6475.239535] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6481.503450] Lustre: server umount lustre-MDT0000 complete [ 6485.029205] LustreError: 336754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780092391 with bad export cookie 7771101246386334022 [ 6485.036265] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6485.422993] Lustre: server umount lustre-MDT0001 complete [ 6497.374991] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 6500.194956] Key type lgssc unregistered [ 6500.476729] LNet: 340048:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6500.483621] LNetError: 340048:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6500.495235] LNet: Removed LNI 192.168.201.160@tcp [ 6501.218252] Key type .llcrypt unregistered [ 6501.225943] Key type ._llcrypt unregistered [ 6529.082954] Key type ._llcrypt registered [ 6529.084705] Key type .llcrypt registered [ 6529.170153] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 6530.195496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6530.243033] alg: No test for adler32 (adler32-zlib) [ 6531.281419] Lustre: Lustre: Build Version: 2.17.53_53_g8b67a55 [ 6531.482475] LNet: Added LNI 192.168.201.160@tcp [8/256/0/180] [ 6533.135201] Key type lgssc registered [ 6534.198818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6541.440371] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6541.465450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6542.902051] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6547.438787] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6553.062817] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6553.083763] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6554.534269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6554.930656] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6558.720344] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6564.271376] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6568.381274] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6574.449899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6574.709620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6580.712201] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6585.173565] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6595.838427] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.60@tcp (stopping) [ 6596.064496] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6596.067900] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6598.624307] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6598.625979] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6598.633831] Lustre: Skipped 1 previous similar message [ 6600.923543] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.60@tcp (stopping) [ 6603.747578] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6603.754084] Lustre: Skipped 1 previous similar message [ 6608.867240] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6608.871069] Lustre: Skipped 2 previous similar messages [ 6609.375132] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6609.498321] Lustre: server umount lustre-MDT0001 complete [ 6611.163344] LustreError: 343219:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6613.988638] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6614.007808] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6616.290664] LustreError: 341600:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6618.079145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6618.148523] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6618.152169] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6618.176994] Lustre: 341594:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6618.374312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6621.586782] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6623.731282] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6624.241979] LustreError: 344849:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6624.252792] LustreError: 344849:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6624.256490] LustreError: 341597:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6624.320671] LustreError: 344852:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.160@tcp: cfg command failed: rc = -17 [ 6624.328440] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6624.328440] [ 6926.568755] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.60@tcp (stopping) [ 6930.912524] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6930.919195] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6930.927631] Lustre: Skipped 1 previous similar message [ 6932.208190] Lustre: server umount lustre-MDT0001 complete [ 6936.039331] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6936.069568] LustreError: 341606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6940.007920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6940.087918] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6940.092908] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6940.120814] Lustre: 341594:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6940.310221] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6943.358455] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 6945.771799] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6947.833596] LustreError: 354514:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6947.846554] LustreError: 354514:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6947.859321] LustreError: 341597:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6947.865097] LustreError: 341597:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6947.912670] LustreError: 354517:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6947.916917] LustreError: 354517:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6947.920249] LustreError: 354517:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.160@tcp: cfg command failed: rc = -17 [ 6947.925984] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.160@tcp [ 6947.925984] [ 7247.840848] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7247.843558] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7247.855152] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7247.863482] Lustre: Skipped 3 previous similar messages [ 7253.169631] Lustre: server umount lustre-MDT0001 complete [ 7254.229715] LustreError: 341601:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7254.240452] LustreError: 341601:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7260.008462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7260.098257] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7260.105726] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7260.129711] Lustre: 341593:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7260.325598] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7262.987213] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 7265.792686] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7269.378676] LustreError: 364472:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7269.386734] LustreError: 364472:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7269.395644] LustreError: 341597:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7269.401618] LustreError: 341597:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7269.439737] LustreError: 364475:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7269.451556] LustreError: 364475:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.160@tcp: cfg command failed: rc = -114 [ 7269.457801] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7269.457801] [ 7583.202135] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7583.204205] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7583.207637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7583.212520] Lustre: Skipped 3 previous similar messages [ 7583.214952] Lustre: Skipped 4 previous similar messages [ 7584.363756] Lustre: server umount lustre-MDT0000 complete [ 7585.938729] LustreError: 341585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780093492 with bad export cookie 5381070960022379022 [ 7585.944169] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7585.947602] LustreError: 341585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7588.323176] LustreError: 341602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7588.332820] LustreError: 341602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7592.127936] Lustre: server umount lustre-MDT0001 complete [ 7593.961189] Lustre: server umount lustre-OST0000 complete [ 7597.962030] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 7602.780145] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 7621.749836] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:25:27 (1780093527) [ 7625.128942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7625.260634] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7625.279275] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7625.319741] Lustre: lustre-MDT0000: new disk, initializing [ 7625.360255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7625.370876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7627.039784] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 7633.212905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7633.274502] Lustre: 376953:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7633.308657] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7633.311450] Lustre: Skipped 1 previous similar message [ 7633.372896] Lustre: lustre-MDT0001: new disk, initializing [ 7633.430476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7633.465492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7633.471257] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7635.420989] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 7640.007760] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7642.092650] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7646.012138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7646.150714] Lustre: lustre-OST0000: new disk, initializing [ 7646.153701] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7646.160336] Lustre: 378693:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7646.210588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7649.278885] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_default_debug -1 all [ 7651.357692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7651.364480] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7651.442310] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7653.744638] Lustre: DEBUG MARKER: oleg160-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7658.976167] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7658.981559] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7658.989859] Lustre: Skipped 1 previous similar message [ 7658.992507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7659.001025] Lustre: Skipped 3 previous similar messages [ 7664.116454] Lustre: server umount lustre-MDT0000 complete [ 7665.967738] LustreError: 378691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780093572 with bad export cookie 5381070960022380968 [ 7665.970032] LustreError: 377742:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7665.970520] LustreError: MGC192.168.201.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7665.976375] LustreError: 378691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7666.135839] Lustre: server umount lustre-MDT0001 complete [ 7678.464349] Lustre: server umount lustre-OST0000 complete [ 7692.852156] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing set_hostid [ 7697.567893] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing load_modules_local [ 7713.004576] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7526 sec ========== 18:26:58 (1780093618) [ 7713.797749] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:26:59 (1780093619) === [ 7715.417931] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:27:01 (1780093621) === [ 7735.196246] Lustre: DEBUG MARKER: oleg160-server.virtnet: executing unload_modules_local [ 7736.546577] Key type lgssc unregistered [ 7736.700660] LNet: 384885:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7736.707393] LNetError: 384885:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7736.717302] LNet: Removed LNI 192.168.201.160@tcp [ 7737.111918] Key type .llcrypt unregistered [ 7737.113637] Key type ._llcrypt unregistered