[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 802178395 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002408] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008019] pid_max: default: 32768 minimum: 301 [ 0.010170] LSM: Security Framework initializing [ 0.011072] Yama: becoming mindful. [ 0.012076] SELinux: Initializing. [ 0.014049] *** VALIDATE selinux *** [ 0.022773] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027802] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028236] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029252] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031049] *** VALIDATE tmpfs *** [ 0.033440] *** VALIDATE proc *** [ 0.035282] *** VALIDATE cgroup *** [ 0.036017] *** VALIDATE cgroup2 *** [ 0.038007] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039219] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041035] Spectre V2 : User space: Vulnerable [ 0.042011] Speculative Store Bypass: Vulnerable [ 0.047028] debug: unmapping init [mem 0xffffffff99059000-0xffffffff99060fff] [ 0.052480] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.053891] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.055036] ... version: 2 [ 0.056025] ... bit width: 48 [ 0.057026] ... generic registers: 4 [ 0.058017] ... value mask: 0000ffffffffffff [ 0.059023] ... max period: 00007fffffffffff [ 0.060024] ... fixed-purpose events: 3 [ 0.061023] ... event mask: 000000070000000f [ 0.062489] rcu: Hierarchical SRCU implementation. [ 0.065991] smp: Bringing up secondary CPUs ... [ 0.068333] x86: Booting SMP configuration: [ 0.070035] .... node #0, CPUs: #1 #2 #3 [ 0.083197] smp: Brought up 1 node, 4 CPUs [ 0.085019] smpboot: Max logical packages: 1 [ 0.086021] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.224195] node 0 deferred pages initialised in 112ms [ 0.232268] devtmpfs: initialized [ 0.234146] x86/mm: Memory block size: 128MB [ 0.239049] gcov: version magic: 0x41383552 [ 0.241376] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.242101] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.244000] pinctrl core: initialized pinctrl subsystem [ 0.246680] [ 0.247018] ************************************************************* [ 0.248026] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.249295] ** ** [ 0.250020] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.251019] ** ** [ 0.252020] ** This means that this kernel is built to expose internal ** [ 0.253022] ** IOMMU data structures, which may compromise security on ** [ 0.254020] ** your system. ** [ 0.256016] ** ** [ 0.257018] ** If you see this message and you are not debugging the ** [ 0.258026] ** kernel, report this immediately to your vendor! ** [ 0.259013] ** ** [ 0.260021] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.261022] ************************************************************* [ 0.262822] NET: Registered protocol family 16 [ 0.263460] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.264134] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.265079] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.268073] cpuidle: using governor menu [ 0.271049] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.273683] PCI: Using configuration type 1 for base access [ 0.276119] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.286059] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.287090] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.288856] cryptd: max_cpu_qlen set to 1000 [ 0.290266] ACPI: Added _OSI(Module Device) [ 0.292023] ACPI: Added _OSI(Processor Device) [ 0.294013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.296015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.303378] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.312752] ACPI: Interpreter enabled [ 0.314205] ACPI: PM: (supports S0 S3 S4 S5) [ 0.317021] ACPI: Using IOAPIC for interrupt routing [ 0.324205] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.346343] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.392399] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.399062] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.402020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.407166] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.412630] acpiphp: Slot [2] registered [ 0.413112] acpiphp: Slot [5] registered [ 0.415126] acpiphp: Slot [6] registered [ 0.417150] acpiphp: Slot [7] registered [ 0.418150] acpiphp: Slot [8] registered [ 0.420168] acpiphp: Slot [9] registered [ 0.422132] acpiphp: Slot [10] registered [ 0.423161] acpiphp: Slot [3] registered [ 0.425111] acpiphp: Slot [4] registered [ 0.426146] acpiphp: Slot [11] registered [ 0.428085] acpiphp: Slot [12] registered [ 0.430138] acpiphp: Slot [13] registered [ 0.431093] acpiphp: Slot [14] registered [ 0.433092] acpiphp: Slot [15] registered [ 0.434081] acpiphp: Slot [16] registered [ 0.436109] acpiphp: Slot [17] registered [ 0.437103] acpiphp: Slot [18] registered [ 0.439171] acpiphp: Slot [19] registered [ 0.440232] acpiphp: Slot [20] registered [ 0.442096] acpiphp: Slot [21] registered [ 0.443092] acpiphp: Slot [22] registered [ 0.445092] acpiphp: Slot [23] registered [ 0.446092] acpiphp: Slot [24] registered [ 0.448128] acpiphp: Slot [25] registered [ 0.449096] acpiphp: Slot [26] registered [ 0.450214] acpiphp: Slot [27] registered [ 0.452103] acpiphp: Slot [28] registered [ 0.453094] acpiphp: Slot [29] registered [ 0.455117] acpiphp: Slot [30] registered [ 0.456105] acpiphp: Slot [31] registered [ 0.458054] PCI host bridge to bus 0000:00 [ 0.459019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.462022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.464026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.466021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.469025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.472027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.474184] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.477110] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.480372] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.489794] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.495539] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.498016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.500014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.502027] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.506623] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.509769] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.512032] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.515877] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.520011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.531978] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.537012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.543674] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.550016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.564019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.576000] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.576000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.576000] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.576000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.614116] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.632399] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.639015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.646017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.661015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.674256] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.685013] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.691057] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.709014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.724000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.729020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.735017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.751017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.760973] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.769000] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.769000] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.769000] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.769000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.769000] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.769000] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.769000] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.769000] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.769402] iommu: Default domain type: Passthrough [ 0.770000] SCSI subsystem initialized [ 0.770000] ACPI: bus type USB registered [ 0.786234] usbcore: registered new interface driver usbfs [ 0.789149] usbcore: registered new interface driver hub [ 0.791164] usbcore: registered new device driver usb [ 0.793164] pps_core: LinuxPPS API ver. 1 registered [ 0.795010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.799184] PTP clock support registered [ 0.802139] EDAC MC: Ver: 3.0.0 [ 0.803914] PCI: Using ACPI for IRQ routing [ 0.806130] NetLabel: Initializing [ 0.807009] NetLabel: domain hash size = 128 [ 0.809056] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.812085] NetLabel: unlabeled traffic allowed by default [ 0.815312] vgaarb: loaded [ 0.816631] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.817007] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.825224] clocksource: Switched to clocksource kvm-clock [ 1.172067] VFS: Disk quotas dquot_6.6.0 [ 1.176495] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.182310] *** VALIDATE ramfs *** [ 1.183912] *** VALIDATE hugetlbfs *** [ 1.186642] pnp: PnP ACPI init [ 1.210718] pnp: PnP ACPI: found 6 devices [ 1.245237] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.250105] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.253204] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.255845] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.259274] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.263660] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.268304] NET: Registered protocol family 2 [ 1.272651] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.280064] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.284828] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.296618] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.302902] TCP: Hash tables configured (established 65536 bind 65536) [ 1.308809] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.318709] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.328579] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.339572] NET: Registered protocol family 1 [ 1.347918] RPC: Registered named UNIX socket transport module. [ 1.352395] RPC: Registered udp transport module. [ 1.381913] RPC: Registered tcp transport module. [ 1.393277] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.400991] NET: Registered protocol family 44 [ 1.404022] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.406373] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.409139] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.411994] PCI: CLS 0 bytes, default 64 [ 1.415396] Unpacking initramfs... [ 3.805651] debug: unmapping init [mem 0xffff94f7bcc54000-0xffff94f7bffbffff] [ 3.810180] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.812462] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.815605] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 6.188782] Initialise system trusted keyrings [ 6.191237] Key type blacklist registered [ 6.204187] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 6.247195] zbud: loaded [ 6.255660] *** VALIDATE nfs *** [ 6.258127] *** VALIDATE nfs4 *** [ 6.263509] pstore: using deflate compression [ 6.269864] Platform Keyring initialized [ 6.521807] NET: Registered protocol family 38 [ 6.527230] Key type asymmetric registered [ 6.533869] Asymmetric key parser 'x509' registered [ 6.538316] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 6.543986] io scheduler mq-deadline registered [ 6.545643] io scheduler kyber registered [ 6.549898] io scheduler bfq registered [ 6.554560] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 6.557838] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 6.562376] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.567858] ACPI: Power Button [PWRF] [ 6.577483] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 6.629437] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 6.721666] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 6.734582] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 6.771797] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.809806] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 6.838748] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.871816] Non-volatile memory driver v1.3 [ 6.876323] Linux agpgart interface v0.103 [ 7.031875] virtio_blk virtio1: [vda] 145792 512-byte logical blocks (74.6 MB/71.2 MiB) [ 7.036574] vda: detected capacity change from 0 to 74645504 [ 7.093213] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 7.096560] vdb: detected capacity change from 0 to 1073741824 [ 7.117893] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.120793] vdc: detected capacity change from 0 to 2621440000 [ 7.136982] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.140101] vdd: detected capacity change from 0 to 2621440000 [ 7.154428] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.157891] vde: detected capacity change from 0 to 4294967296 [ 7.214485] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.219790] vdf: detected capacity change from 0 to 4294967296 [ 7.229660] libphy: Fixed MDIO Bus: probed [ 7.239754] usbcore: registered new interface driver usbserial_generic [ 7.253925] usbserial: USB Serial support registered for generic [ 7.256425] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 7.264884] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 7.266653] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 7.268984] mousedev: PS/2 mouse device common for all mice [ 7.272116] rtc_cmos 00:05: RTC can wake from S4 [ 7.275102] rtc_cmos 00:05: registered as rtc0 [ 7.277170] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 7.279894] intel_pstate: CPU model not supported [ 7.284327] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 7.290568] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 7.295195] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 7.303558] hid: raw HID events driver (C) Jiri Kosina [ 7.306608] usbcore: registered new interface driver usbhid [ 7.308982] usbhid: USB HID core driver [ 7.310890] drop_monitor: Initializing network drop monitor service [ 7.313971] Initializing XFRM netlink socket [ 7.316398] NET: Registered protocol family 10 [ 7.319844] Segment Routing with IPv6 [ 7.321338] NET: Registered protocol family 17 [ 7.323525] mpls_gso: MPLS GSO support [ 7.328651] RAS: Correctable Errors collector initialized. [ 7.330820] AVX version of gcm_enc/dec engaged. [ 7.332456] AES CTR mode by8 optimization enabled [ 7.515621] sched_clock: Marking stable (7515506892, 0)->(9534477619, -2018970727) [ 7.520917] registered taskstats version 1 [ 7.531449] Loading compiled-in X.509 certificates [ 7.536552] zswap: loaded using pool lzo/zbud [ 7.579859] Key type big_key registered [ 7.595910] Key type encrypted registered [ 7.598509] ima: No TPM chip found, activating TPM-bypass! [ 7.602147] ima: Allocated hash algorithm: sha1 [ 7.604796] ima: No architecture policies found [ 7.607515] evm: Initialising EVM extended attributes: [ 7.610490] evm: security.selinux [ 7.612374] evm: security.ima [ 7.614258] evm: security.capability [ 7.616492] evm: HMAC attrs: 0x1 [ 7.619912] rtc_cmos 00:05: setting system clock to 2026-07-15 13:57:28 UTC (1784123848) [ 7.635864] debug: unmapping init [mem 0xffffffff9a003000-0xffffffff9a1fffff] [ 7.648797] debug: unmapping init [mem 0xffffffff98d82000-0xffffffff99058fff] [ 7.662662] Write protecting the kernel read-only data: 28672k [ 7.669767] debug: unmapping init [mem 0xffffffff97403000-0xffffffff975fffff] [ 7.676515] debug: unmapping init [mem 0xffffffff97d14000-0xffffffff97dfffff] [ 7.750376] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.777410] systemd[1]: Detected virtualization kvm. [ 7.783975] systemd[1]: Detected architecture x86-64. [ 7.793669] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.982742] systemd[1]: No hostname configured. [ 7.986414] systemd[1]: Set hostname to . [ 7.995102] random: systemd: uninitialized urandom read (16 bytes read) [ 8.001745] systemd[1]: Initializing machine ID from random generator. [ 8.210629] random: ln: uninitialized urandom read (6 bytes read) [ 8.341456] random: systemd: uninitialized urandom read (16 bytes read) [ 8.344443] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 8.355147] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 8.364745] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 10.145405] device-mapper: uevent: version 1.0.3 [ 10.148500] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 13.462326] random: fast init done [ 14.290834] virtio_net virtio0 ens2: renamed from eth0 [ 14.805324] scsi host0: ata_piix [ 15.056591] scsi host1: ata_piix [ 15.093576] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 15.102551] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 16.184656] hrtimer: interrupt took 9033453 ns [ 20.219064] random: crng init done [ 20.225481] random: 7 urandom warning(s) missed due to ratelimiting [ 23.457942] dracut-initqueue[584]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 25.633699] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 29.140496] printk: systemd: 26 output lines suppressed due to ratelimiting [ 30.678221] SELinux: Disabled at runtime. [ 30.865358] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 30.875456] systemd[1]: Detected virtualization kvm. [ 30.879268] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 33.474333] systemd[1]: initrd-switch-root.service: Succeeded. [ 33.478341] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 33.548478] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 33.570337] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 33.703218] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 33.772110] systemd[1]: Starting Journal Service... Starting Journal Service... [ 33.846709] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. [ 34.649625] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 36.968515] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 38.637627] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 38.859910] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 39.986984] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 40.395046] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…only root support (10s / no limit) [*** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (11s / no limit)[ 44.865914] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (11s / no limit) [ ***] A start job is running for Configur…only root support (12s / no limit)[ 46.065363] NFS: Registering the id_resolver key type [ 46.070259] Key type id_resolver registered [ 46.072132] Key type id_legacy registered [ **] A start job is running for Configur…only root support (12s / no limit) [ *] A start job is running for Configur…only root support (13s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg339-server login: [ 81.360350] libcfs: loading out-of-tree module taints kernel. [ 81.380389] Key type ._llcrypt registered [ 81.382125] Key type .llcrypt registered [ 81.430310] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 90.367753] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 91.020159] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 91.028159] alg: No test for adler32 (adler32-zlib) [ 92.157139] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 92.620875] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 94.271222] Key type lgssc registered [ 95.003810] Lustre: Echo OBD driver; http://www.lustre.org/ [ 103.578410] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 123.788684] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 129.726962] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 129.738067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 130.851213] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 130.865231] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 130.910794] Lustre: lustre-MDT0000: new disk, initializing [ 130.955791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 130.976365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 132.746246] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 138.758776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 138.802261] Lustre: 6481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 138.822454] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 138.824812] Lustre: Skipped 1 previous similar message [ 138.873094] Lustre: lustre-MDT0001: new disk, initializing [ 138.908309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 138.920230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 138.926950] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 140.641382] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 143.335151] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 147.694121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 147.807486] Lustre: lustre-OST0000: new disk, initializing [ 147.809546] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 147.812853] Lustre: 8418:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 147.834291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 147.876031] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 147.879962] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 147.901232] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 150.254630] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 157.225092] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 157.295215] Lustre: lustre-OST0001: new disk, initializing [ 157.297559] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 157.301195] Lustre: 9490:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 157.328943] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 160.378626] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 163.834183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 163.840551] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 163.874312] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 167.359129] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 172.616451] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 175.564533] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing check_logdir /tmp/testlogs/ [ 177.686924] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing yml_node [ 179.924241] Lustre: DEBUG MARKER: Client: 2.17.54.163 [ 181.091194] Lustre: DEBUG MARKER: MDS: 2.17.54.163 [ 182.246475] Lustre: DEBUG MARKER: OSS: 2.17.54.163 [ 183.014168] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jul 15 10:00:23 EDT 2026 [ 190.968481] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 191.627556] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 204.768752] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 204.771232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 204.776264] Lustre: Skipped 2 previous similar messages [ 204.780187] Lustre: Skipped 1 previous similar message [ 209.888117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 209.894029] Lustre: Skipped 5 previous similar messages [ 210.175247] Lustre: server umount lustre-MDT0000 complete [ 212.173967] LustreError: 6474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124053 with bad export cookie 2401371919455644059 [ 212.177557] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 212.182631] LustreError: 6474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 215.011538] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 215.012533] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 215.020156] Lustre: Skipped 2 previous similar messages [ 215.028864] Lustre: Skipped 1 previous similar message [ 217.057352] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 217.061592] Lustre: Skipped 1 previous similar message [ 218.411571] Lustre: server umount lustre-MDT0001 complete [ 220.341832] Lustre: server umount lustre-OST0000 complete [ 222.427805] Lustre: server umount lustre-OST0001 complete [ 225.546905] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 230.286457] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 249.140348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 249.267192] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 249.296614] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 249.350900] Lustre: lustre-MDT0000: new disk, initializing [ 249.397773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 249.417960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 251.482080] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 257.357511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 257.426610] Lustre: 15044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 257.459370] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 257.462410] Lustre: Skipped 1 previous similar message [ 257.505968] Lustre: lustre-MDT0001: new disk, initializing [ 257.557416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 257.574056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 257.581279] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 259.726805] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 265.110207] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 268.022420] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 271.473973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 271.636216] Lustre: 17009:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 273.610477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 273.616901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 273.694068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 274.903763] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 280.490520] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 282.885097] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 282.992844] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 285.045811] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 285.139041] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 289.254327] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 289.254939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 289.255832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 289.270629] Lustre: Skipped 1 previous similar message [ 292.230725] Lustre: server umount lustre-OST0000 complete [ 298.464570] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 298.470440] LustreError: Skipped 1 previous similar message [ 298.474945] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 298.490733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 298.495417] Lustre: Skipped 1 previous similar message [ 301.362443] Lustre: server umount lustre-MDT0000 complete [ 303.663887] LustreError: 15035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124144 with bad export cookie 2401371919455645578 [ 303.666647] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 303.668765] LustreError: 15035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 303.848874] Lustre: server umount lustre-MDT0001 complete [ 307.161037] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:02:26 (1784124146) [ 310.884638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 311.102359] LustreError: 19271:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 311.114744] LustreError: 19271:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 311.145433] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 311.149686] Lustre: Skipped 1 previous similar message [ 313.453680] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 316.388513] LustreError: 19272:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 318.175289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 320.541867] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 324.321301] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 327.077925] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 330.432928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 333.290820] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 336.722600] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 348.271412] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 354.273447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 354.274943] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.276320] LustreError: 21761:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 354.293990] Lustre: Skipped 3 previous similar messages [ 354.306530] Lustre: server umount lustre-MDT0000 complete [ 356.177932] LustreError: 19253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124197 with bad export cookie 2401371919455646460 [ 356.184251] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 356.186575] LustreError: 19253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 358.362807] Lustre: server umount lustre-MDT0001 complete [ 370.625311] Lustre: server umount lustre-OST0000 complete [ 381.545311] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 387.161606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 387.375024] LustreError: 23670:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 387.384149] LustreError: 23670:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 387.416042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 387.423806] Lustre: Skipped 2 previous similar messages [ 389.688603] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 392.673095] LustreError: 23671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 394.004555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 396.131980] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 397.492969] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 400.891834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 404.387816] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 409.164680] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 409.255418] Lustre: lustre-OST0001: new disk, initializing [ 409.257506] Lustre: Skipped 1 previous similar message [ 409.260042] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 409.263403] Lustre: Skipped 1 previous similar message [ 409.265867] Lustre: 25985:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 410.517693] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 410.522179] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 410.563437] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 412.615212] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 418.609599] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 420.599379] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 428.004495] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 428.007175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 428.018434] Lustre: Skipped 3 previous similar messages [ 428.026107] Lustre: Skipped 5 previous similar messages [ 429.281776] Lustre: server umount lustre-MDT0000 complete [ 430.974048] LustreError: 23650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124271 with bad export cookie 2401371919455647307 [ 430.977482] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 430.980727] LustreError: 23650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 433.122630] LustreError: 23665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 433.131677] LustreError: 23665:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 433.198191] Lustre: server umount lustre-MDT0001 complete [ 435.267733] Lustre: server umount lustre-OST0000 complete [ 437.307139] Lustre: server umount lustre-OST0001 complete [ 445.024341] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 449.236076] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 449.427429] LustreError: 28996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 451.128190] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 454.870460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 455.099674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 455.103970] Lustre: Skipped 4 previous similar messages [ 456.969671] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 458.385981] Lustre: 30142:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 461.403942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 464.136328] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 467.805332] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 470.419420] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 479.627403] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 481.235578] Lustre: 32020:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 485.349933] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.352263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 485.360546] Lustre: Skipped 3 previous similar messages [ 485.366629] Lustre: Skipped 3 previous similar messages [ 489.477593] Lustre: server umount lustre-MDT0000 complete [ 490.467208] LustreError: 28997:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 490.474489] LustreError: 28997:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 491.208863] LustreError: 28976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124332 with bad export cookie 2401371919455648602 [ 491.211305] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 491.214290] LustreError: 28976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 491.351553] Lustre: server umount lustre-MDT0001 complete [ 502.790287] Lustre: server umount lustre-OST0000 complete [ 514.598600] Lustre: server umount lustre-OST0001 complete [ 518.110883] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:05:58 (1784124358) [ 529.295240] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 533.696948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 536.059920] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 540.183756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.279952] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 543.660525] Lustre: 35963:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 546.692173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 549.738686] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 553.585669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 556.395554] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 565.470230] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 566.958152] Lustre: 37841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 569.317823] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 569.319461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 569.326107] Lustre: Skipped 5 previous similar messages [ 569.328850] Lustre: Skipped 2 previous similar messages [ 574.900870] Lustre: server umount lustre-MDT0000 complete [ 576.601561] LustreError: 34797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124417 with bad export cookie 2401371919455649701 [ 576.607511] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 576.609145] LustreError: 34797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 576.760707] Lustre: server umount lustre-MDT0001 complete [ 588.807605] Lustre: server umount lustre-OST0000 complete [ 600.754472] Lustre: server umount lustre-OST0001 complete [ 604.740955] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:07:24 (1784124444) [ 611.697663] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 616.496470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 616.723915] LustreError: 39978:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 616.736153] LustreError: 39978:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 616.762100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 616.768602] Lustre: Skipped 6 previous similar messages [ 618.667394] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 622.811076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 624.700778] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 626.028644] Lustre: 41126:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 628.955401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 631.849278] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 635.953426] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 640.147399] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 647.896995] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 658.290965] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 658.296759] Lustre: Skipped 1 previous similar message [ 662.503114] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 662.524301] Lustre: Skipped 2 previous similar messages [ 664.569981] Lustre: server umount lustre-MDT0000 complete [ 671.478284] LustreError: 43060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124512 with bad export cookie 2401371919455650800 [ 671.489727] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 671.497197] LustreError: 43060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 677.866776] Lustre: server umount lustre-MDT0001 complete [ 688.715955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 702.111434] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 706.217013] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 708.257723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 711.903460] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 713.712116] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 713.721915] Lustre: Skipped 1 previous similar message [ 717.622602] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 720.980743] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 2 sec [ 725.015810] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 726.069544] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 734.177774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 734.193673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 734.197904] Lustre: Skipped 18 previous similar messages [ 735.360266] Lustre: server umount lustre-MDT0000 complete [ 737.658193] LustreError: 39959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124578 with bad export cookie 2401371919455651990 [ 737.665190] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 737.665590] LustreError: 39959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 737.868228] Lustre: server umount lustre-MDT0001 complete [ 750.571997] Lustre: server umount lustre-OST0000 complete [ 763.285558] Lustre: server umount lustre-OST0001 complete [ 769.042256] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:10:08 (1784124608) [ 778.958240] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 785.025567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 785.235271] LustreError: 47618:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 785.247920] LustreError: 47618:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 787.833686] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 793.459510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 796.287069] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 798.158790] Lustre: 48770:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 798.166163] Lustre: 48770:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 802.658501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 807.015992] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 812.549353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 816.647135] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 821.733746] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 824.867477] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 833.511346] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 833.534993] Lustre: Skipped 9 previous similar messages [ 839.463786] Lustre: server umount lustre-MDT0000 complete [ 842.156615] LustreError: 47598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784124683 with bad export cookie 2401371919455652788 [ 842.165883] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 842.166714] LustreError: 47598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 842.415540] Lustre: server umount lustre-MDT0001 complete [ 855.499771] Lustre: server umount lustre-OST0000 complete [ 867.449276] Lustre: server umount lustre-OST0001 complete [ 873.467452] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:11:53 (1784124713) [ 875.044652] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 876.113420] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:11:55 (1784124715) [ 883.081119] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 888.784419] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 913.100554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 913.279622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 913.336834] Lustre: lustre-MDT0000: new disk, initializing [ 913.446806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 913.452235] Lustre: Skipped 9 previous similar messages [ 913.464302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 918.860567] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 932.488688] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 942.173924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 942.395083] Lustre: 54846:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 943.679201] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 943.713056] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 950.579559] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 961.223994] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 963.916103] Lustre: DEBUG MARKER: create test files [ 989.668721] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 994.788249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 994.794232] Lustre: Skipped 9 previous similar messages [ 995.421236] Lustre: server umount lustre-OST0000 complete [ 998.196235] Lustre: server umount lustre-MDT0000 complete [ 1005.279294] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1007.257878] Key type lgssc unregistered [ 1007.486340] LNet: 56476:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1007.493735] LNetError: 56476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1007.513606] LNet: Removed LNI 192.168.203.139@tcp [ 1008.166154] Key type .llcrypt unregistered [ 1008.170894] Key type ._llcrypt unregistered [ 1026.713353] Key type ._llcrypt registered [ 1026.716470] Key type .llcrypt registered [ 1026.795202] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1027.498708] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1027.522517] alg: No test for adler32 (adler32-zlib) [ 1028.503272] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 1028.633604] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1030.271674] Key type lgssc registered [ 1031.005662] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1036.189124] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1036.208246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1037.485740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1040.149224] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1045.407734] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1049.860090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1050.087297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1053.913625] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1058.171701] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1062.311815] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1062.353586] Lustre: 59186:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1062.359303] Lustre: 59186:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1062.364554] Lustre: 59186:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1062.369608] Lustre: 59186:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1062.379929] Lustre: 59186:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1062.386971] Lustre: 59186:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1067.713517] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1071.502426] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1077.728224] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1077.737856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1077.746309] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1081.299911] Lustre: server umount lustre-OST0000 complete [ 1083.826185] Lustre: server umount lustre-MDT0000 complete [ 1090.523655] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1092.407904] Key type lgssc unregistered [ 1092.616652] LNet: 60795:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1092.621925] LNetError: 60795:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1092.638344] LNet: Removed LNI 192.168.203.139@tcp [ 1093.263476] Key type .llcrypt unregistered [ 1093.267773] Key type ._llcrypt unregistered [ 1111.864520] Key type ._llcrypt registered [ 1111.866874] Key type .llcrypt registered [ 1111.927483] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1112.478023] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1112.496641] alg: No test for adler32 (adler32-zlib) [ 1113.422832] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 1113.564378] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1115.199211] Key type lgssc registered [ 1115.909579] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1120.670514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1120.689704] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1121.935384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1124.395861] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1128.595846] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1132.642282] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1132.864189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1136.425862] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1140.503365] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1147.113372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1147.155264] Lustre: 63606:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1147.161162] Lustre: 63606:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1147.165451] Lustre: 63606:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1147.173818] Lustre: 63606:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1147.178092] Lustre: 63606:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1147.182125] Lustre: 63606:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1154.501278] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1162.720241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1162.722635] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1162.729182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1164.708353] Lustre: server umount lustre-OST0000 complete [ 1166.608687] Lustre: server umount lustre-MDT0000 complete [ 1171.614598] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 1173.348251] Key type lgssc unregistered [ 1173.521423] LNet: 65186:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1173.535405] LNetError: 65186:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1173.553666] LNet: Removed LNI 192.168.203.139@tcp [ 1174.076782] Key type .llcrypt unregistered [ 1174.079093] Key type ._llcrypt unregistered [ 1185.145104] Key type ._llcrypt registered [ 1185.147519] Key type .llcrypt registered [ 1185.221449] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1194.423779] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1195.106149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1195.121381] alg: No test for adler32 (adler32-zlib) [ 1196.066618] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 1196.229102] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 1197.847142] Key type lgssc registered [ 1198.566696] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1219.421683] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1219.442674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.547866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1220.569168] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1220.619345] Lustre: lustre-MDT0000: new disk, initializing [ 1220.678178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1220.691261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1223.465766] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1230.480131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1230.538685] Lustre: 68313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1230.574092] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1230.578095] Lustre: Skipped 1 previous similar message [ 1230.627712] Lustre: lustre-MDT0001: new disk, initializing [ 1230.691806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1230.717125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1230.727643] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1233.094222] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1239.212358] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1242.538505] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1246.678222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1246.836488] Lustre: lustre-OST0000: new disk, initializing [ 1246.840712] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1246.847805] Lustre: 70279:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1246.906526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1250.118875] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1253.926917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1253.933357] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1254.017069] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1256.007779] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1258.496233] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1258.613903] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1261.240737] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1261.348112] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1264.099731] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1264.102216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1264.103583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1264.121132] Lustre: Skipped 1 previous similar message [ 1268.593838] Lustre: server umount lustre-OST0000 complete [ 1271.777108] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1271.787705] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1271.799260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1271.805034] Lustre: Skipped 1 previous similar message [ 1274.336153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1274.337323] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1277.140300] Lustre: server umount lustre-MDT0000 complete [ 1279.574614] LustreError: 69108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125120 with bad export cookie 6855587148814744577 [ 1279.585548] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1279.587487] LustreError: 69108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1279.824807] Lustre: server umount lustre-MDT0001 complete [ 1285.037640] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:18:44 (1784125124) [ 1291.064865] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1311.638600] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1316.293591] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1335.452055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1335.581122] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1335.600820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1335.651599] Lustre: lustre-MDT0000: new disk, initializing [ 1335.695213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1335.710863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1337.965438] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1343.974437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1344.061925] Lustre: 76289:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1344.104668] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1344.107584] Lustre: Skipped 1 previous similar message [ 1344.162464] Lustre: lustre-MDT0001: new disk, initializing [ 1344.217638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1344.245428] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1344.260493] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1346.791266] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1352.361859] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1355.281552] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1358.939690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1359.093638] Lustre: lustre-OST0000: new disk, initializing [ 1359.096190] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1359.099860] Lustre: 78255:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1359.149659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1362.716459] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1368.561431] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1369.102790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1369.109994] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1369.181341] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1370.934210] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1371.066702] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1373.779547] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1373.889963] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1379.301269] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1379.305486] LustreError: Skipped 1 previous similar message [ 1379.305500] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1379.308251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1379.320217] Lustre: Skipped 1 previous similar message [ 1379.328064] Lustre: Skipped 2 previous similar messages [ 1381.062535] Lustre: server umount lustre-OST0000 complete [ 1384.419572] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1384.420994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1384.428210] Lustre: Skipped 1 previous similar message [ 1384.440023] Lustre: Skipped 1 previous similar message [ 1389.423678] Lustre: server umount lustre-MDT0000 complete [ 1389.538823] LustreError: 76294:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1389.557780] LustreError: 76294:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 1391.815496] LustreError: 76280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125232 with bad export cookie 6855587148814745452 [ 1391.818362] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1391.827521] LustreError: 76280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1392.039231] Lustre: server umount lustre-MDT0001 complete [ 1396.876551] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:20:36 (1784125236) [ 1400.813884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1401.121125] LustreError: 80663:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1401.168279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1403.448423] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1406.434295] LustreError: 80664:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1408.062024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1410.555838] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1414.904073] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1417.940635] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1422.000800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1425.488120] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1429.451404] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1431.304692] Lustre: 83331:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 7380918f-2420-4088-9c37-e5b7e6c98b19 at adminstrative request [ 1433.446904] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1433.459973] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1443.809491] Lustre: MGS: Client 7380918f-2420-4088-9c37-e5b7e6c98b19 (at 0@lo) reconnecting [ 1443.817127] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5f23f0307bd602e5 to 0x5f23f0307bd60585 [ 1443.826208] Lustre: MGC192.168.203.139@tcp: Connection restored to 0@lo (at 0@lo) [ 1465.307885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1465.409719] Lustre: lustre-OST0001: new disk, initializing [ 1465.413626] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1465.418240] Lustre: 83725:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1465.466196] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1465.470045] Lustre: Skipped 2 previous similar messages [ 1468.492658] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1473.417061] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1473.559170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1473.565041] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1473.628739] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1478.628980] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1478.634963] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1478.636698] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1478.645839] Lustre: Skipped 1 previous similar message [ 1481.190990] Lustre: server umount lustre-OST0001 complete [ 1483.748490] LustreError: 82919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1483.759745] LustreError: 82919:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1488.867946] LustreError: 82483:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1488.874832] LustreError: 82483:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1489.393251] Lustre: server umount lustre-OST0000 complete [ 1499.104871] LustreError: 80663:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1499.113741] LustreError: 80663:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1499.516742] Lustre: server umount lustre-MDT0000 complete [ 1501.245316] LustreError: 83736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125342 with bad export cookie 6855587148814747013 [ 1501.248940] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1501.251144] LustreError: 83736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1501.420794] Lustre: server umount lustre-MDT0001 complete [ 1505.552441] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:22:25 (1784125345) [ 1506.473506] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1507.372774] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:22:27 (1784125347) [ 1508.240419] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1509.113778] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:22:29 (1784125349) [ 1509.945807] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1510.742986] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:22:30 (1784125350) [ 1511.483200] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1512.334857] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:22:32 (1784125352) [ 1513.102464] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1516.520592] loop: module loaded [ 1516.527553] loop0: detected capacity change from 0 to 4294967296 [ 1516.549344] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1517.361231] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1519.086778] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1521.078234] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1535.460292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1535.499358] LustreError: 87841:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1535.503113] LustreError: 87841:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1535.507587] LustreError: 87841:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1535.511487] LustreError: 87841:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1545.580718] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1550.420429] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1569.788625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1569.921302] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1569.950288] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1570.002762] Lustre: lustre-MDT0000: new disk, initializing [ 1570.049337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1570.065368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1572.312878] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1578.487740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1578.558385] Lustre: 90700:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1578.706502] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1580.843455] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1586.491191] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1589.457964] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1593.092760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1593.244296] Lustre: 92668:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1596.285360] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1599.517482] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1599.582785] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1601.966291] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1604.087743] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1604.202463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1606.715124] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1606.852914] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1609.697170] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1609.697645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1609.707247] LustreError: Skipped 2 previous similar messages [ 1609.713316] Lustre: Skipped 4 previous similar messages [ 1609.717211] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1609.724906] Lustre: Skipped 8 previous similar messages [ 1614.068455] Lustre: server umount lustre-OST0000 complete [ 1619.937657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1622.117691] Lustre: server umount lustre-MDT0000 complete [ 1623.940673] LustreError: 90695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125464 with bad export cookie 6855587148814747629 [ 1623.943603] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1623.946230] LustreError: 90695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1624.109046] Lustre: server umount lustre-MDT0001 complete [ 1628.363829] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:24:28 (1784125468) [ 1631.909115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1632.151174] LustreError: 95078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1634.217808] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1638.670729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1641.000923] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1644.458337] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1647.113330] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1650.594544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1653.705260] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1657.512860] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1659.761191] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1659.880869] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1662.220718] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1665.447537] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 3 sec [ 1669.195544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1669.273653] Lustre: lustre-OST0001: new disk, initializing [ 1669.275909] Lustre: Skipped 2 previous similar messages [ 1669.278742] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1669.283564] Lustre: Skipped 3 previous similar messages [ 1669.286601] Lustre: 98466:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1672.812292] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1674.789060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1674.795626] Lustre: Skipped 2 previous similar messages [ 1674.799902] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1674.848392] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1678.337093] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1680.505406] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1680.639721] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1682.753336] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1682.861240] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1695.505169] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1709.572830] Lustre: Failing over lustre-MDT0000 [ 1709.867760] Lustre: server umount lustre-MDT0000 complete [ 1710.085236] LustreError: 95073:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1710.098941] LustreError: 95073:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1710.563727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1710.564494] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1710.578041] Lustre: Skipped 4 previous similar messages [ 1726.943102] Lustre: 66541:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784125551/real 1784125551] req@00000000d764a144 x1870790695014272/t0(0) o400->MGC192.168.203.139@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1784125567 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1726.956169] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1727.172879] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1727.176387] LDISKFS-fs (dm-0): recovery complete [ 1727.183418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1736.160178] LustreError: 66538:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000077af6982 x1870790695022208/t0(0) o250->MGC192.168.203.139@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1736.285423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1736.288550] Lustre: Skipped 6 previous similar messages [ 1736.320287] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1738.394964] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1740.783053] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1740.783240] Lustre: 101292:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1741.287131] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1741.295636] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1741.300752] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 2 previous similar messages [ 1741.315382] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1741.631341] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1741.954677] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1741.962132] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1742.287128] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1742.291074] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1742.302154] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1742.311668] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1743.259849] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1743.267510] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1743.583114] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1743.587479] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1744.531588] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1744.538351] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1745.480047] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1745.489846] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1745.807278] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1745.813346] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1748.631932] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1748.639321] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 1749.585564] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1749.595201] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1749.911203] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1749.914864] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1756.791940] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1756.799658] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 1757.753727] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1757.763511] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1758.079376] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1758.083915] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1772.835862] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1772.842950] Lustre: 101316:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 1773.783937] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1773.789951] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1774.103302] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1774.107673] LustreError: 101316:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1800.103149] Lustre: 101316:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1800.108264] Lustre: 101316:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1800.114142] Lustre: 101316:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ad09a160-b4bf-495c-ae08-69f962627c80@192.168.203.39@tcp [ 1800.122899] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1800.126437] Lustre: 101316:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@0000000073c1a901 x1870790684382464/t0(8589934784) o35->ad09a160-b4bf-495c-ae08-69f962627c80@192.168.203.39@tcp:251/0 lens 392/0 e 0 to 0 dl 1784125651 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1800.140694] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1800.148410] Lustre: lustre-MDT0000: Denying connection for new client ad09a160-b4bf-495c-ae08-69f962627c80 (at 192.168.203.39@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1800.153867] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1800.175208] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1800.177054] Lustre: Skipped 2 previous similar messages [ 1800.179654] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1800.200630] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1800.201931] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1807.848504] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1807.850801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1807.855129] Lustre: Skipped 2 previous similar messages [ 1807.862992] Lustre: Skipped 3 previous similar messages [ 1822.177283] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1822.234802] Lustre: server umount lustre-OST0000 complete [ 1823.205350] LustreError: 96898:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1823.221083] LustreError: 96898:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 39 previous similar messages [ 1826.272944] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1830.134938] Lustre: server umount lustre-OST0001 complete [ 1846.240322] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1846.352440] Lustre: server umount lustre-MDT0000 complete [ 1848.134575] LustreError: 96920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125689 with bad export cookie 6855587148814798834 [ 1848.137919] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1848.141443] LustreError: 96920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1848.306949] Lustre: server umount lustre-MDT0001 complete [ 1852.057918] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:28:12 (1784125692) [ 1855.768326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1855.771722] Lustre: *** cfs_fail_loc=197, val=0*** [ 1871.263598] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1876.383701] LustreError: 103290:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.139@tcp: failed processing log, type 4: rc = -110 [ 1881.525649] Lustre: *** cfs_fail_loc=197, val=0*** [ 1881.527775] Lustre: Skipped 65 previous similar messages [ 1904.806440] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1907.985303] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1908.985659] Lustre: server umount lustre-OST0000 complete [ 1912.967423] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:29:12 (1784125752) [ 1928.192781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1930.300213] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1935.269856] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1937.754101] LustreError: 104940:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1937.760763] LustreError: 104940:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1937.765188] LustreError: 104940:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -5 [ 1937.921333] Lustre: server umount lustre-MDT0000 complete [ 1953.488243] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 1958.210227] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 1975.917358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1976.036501] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1976.052213] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1976.093642] Lustre: lustre-MDT0000: new disk, initializing [ 1976.144585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1978.279697] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1984.057307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1984.121423] Lustre: 109108:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1984.259661] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1986.378352] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 1991.118378] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1993.829322] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1997.126545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1997.270764] Lustre: 111074:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1997.311542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1997.315453] Lustre: Skipped 4 previous similar messages [ 2000.165462] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2004.549405] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2005.310524] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2007.487838] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2007.583311] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2009.909791] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2010.017731] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2014.690767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2014.692885] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2014.698257] Lustre: Skipped 4 previous similar messages [ 2014.703347] LustreError: Skipped 1 previous similar message [ 2014.707341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2014.711433] Lustre: Skipped 14 previous similar messages [ 2017.130227] Lustre: server umount lustre-OST0000 complete [ 2025.189429] Lustre: server umount lustre-MDT0000 complete [ 2027.182106] LustreError: 109099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125868 with bad export cookie 6855587148814809572 [ 2027.182844] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2027.190673] LustreError: 109099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2027.362309] Lustre: server umount lustre-MDT0001 complete [ 2031.787544] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:31:11 (1784125871) [ 2036.330484] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2037.837726] Key type lgssc unregistered [ 2037.994531] LNet: 113560:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2038.001380] LNetError: 113560:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2038.013357] LNet: Removed LNI 192.168.203.139@tcp [ 2038.534401] Key type .llcrypt unregistered [ 2038.536943] Key type ._llcrypt unregistered [ 2047.908801] Key type ._llcrypt registered [ 2047.911302] Key type .llcrypt registered [ 2047.973533] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 2056.352942] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2056.793973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2056.863084] alg: No test for adler32 (adler32-zlib) [ 2057.787763] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2057.928237] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2059.527187] Key type lgssc registered [ 2060.112697] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2081.225346] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2081.239783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2082.355721] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2082.373815] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2082.418323] Lustre: lustre-MDT0000: new disk, initializing [ 2082.461263] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2082.472688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2084.322343] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2090.295578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2090.432552] Lustre: lustre-OST0000: new disk, initializing [ 2090.436062] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2090.439087] Lustre: Skipped 1 previous similar message [ 2090.442404] Lustre: 117707:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2090.479337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2091.287805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2091.292435] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2091.313572] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2093.258431] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2098.702197] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2098.781857] Lustre: lustre-OST0001: new disk, initializing [ 2098.785277] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2098.789285] Lustre: 118570:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2098.851174] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2101.736199] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2105.882384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2105.887906] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2105.902689] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2106.814600] Lustre: server umount lustre-MDT0000 complete [ 2109.830279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2111.964257] LustreError: 116895:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784125952 with bad export cookie 11536137544299352281 [ 2111.972514] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2121.237460] Lustre: server umount lustre-OST0000 complete [ 2133.186256] Lustre: server umount lustre-OST0001 complete [ 2137.034246] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:32:57 (1784125977) [ 2146.671944] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:33:06 (1784125986) [ 2150.795547] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2160.320117] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:33:20 (1784126000) [ 2178.338487] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 2182.900494] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2198.430055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2198.545938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2198.562268] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2198.618660] Lustre: lustre-MDT0000: new disk, initializing [ 2198.659254] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2198.667543] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2200.650936] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2206.049116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2206.107131] Lustre: 126306:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2206.145155] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2206.148544] Lustre: Skipped 1 previous similar message [ 2206.193715] Lustre: lustre-MDT0001: new disk, initializing [ 2206.256802] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2206.282059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2206.287652] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2208.095440] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2212.880036] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2215.349714] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2218.486205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2218.631878] Lustre: lustre-OST0000: new disk, initializing [ 2218.634590] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2218.638650] Lustre: 128274:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2218.682444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2221.063638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2221.068529] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2221.109430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2221.368796] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2226.039411] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2228.006958] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2228.101695] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2229.852360] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2229.947984] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2231.265244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2231.266430] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2231.269352] LustreError: Skipped 1 previous similar message [ 2231.276430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2231.281142] Lustre: Skipped 1 previous similar message [ 2236.388384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2236.393279] Lustre: Skipped 1 previous similar message [ 2241.507043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2241.511854] Lustre: Skipped 1 previous similar message [ 2245.087397] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2245.163886] Lustre: server umount lustre-OST0000 complete [ 2247.137899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2247.142821] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2247.149525] Lustre: Skipped 1 previous similar message [ 2247.152109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2251.744857] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2251.745385] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2253.125327] Lustre: server umount lustre-MDT0000 complete [ 2255.005653] LustreError: 127101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126095 with bad export cookie 11536137544299353485 [ 2255.008507] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2255.015310] LustreError: 127101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2255.177805] Lustre: server umount lustre-MDT0001 complete [ 2259.361302] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:34:59 (1784126099) [ 2262.426772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2262.704599] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2264.527207] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2268.131319] LustreError: 130687:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2268.144890] LustreError: 130687:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2268.646386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2270.683915] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2273.900547] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2276.386701] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2279.481956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2282.174873] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2285.326305] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2296.689267] Lustre: *** cfs_fail_loc=159, val=0*** [ 2313.217742] Lustre: lustre-MDT0000: Client ac8561ba-1cb5-4959-84ca-11a33e2888ff (at 192.168.203.39@tcp) reconnecting [ 2313.835396] Lustre: *** cfs_fail_loc=159, val=0*** [ 2313.837731] Lustre: Skipped 5 previous similar messages [ 2330.113422] Lustre: lustre-MDT0000: Client ac8561ba-1cb5-4959-84ca-11a33e2888ff (at 192.168.203.39@tcp) reconnecting [ 2345.443222] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2345.445114] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2345.448670] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2345.454526] Lustre: Skipped 1 previous similar message [ 2345.461052] Lustre: Skipped 2 previous similar messages [ 2351.258658] Lustre: server umount lustre-OST0000 complete [ 2355.680811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2359.006669] Lustre: server umount lustre-MDT0000 complete [ 2360.630501] LustreError: 130667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126201 with bad export cookie 11536137544299354367 [ 2360.633101] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2360.639373] LustreError: 130667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2360.785589] Lustre: server umount lustre-MDT0001 complete [ 2365.276678] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2366.551749] Key type lgssc unregistered [ 2366.712093] LNet: 134479:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2366.717079] LNetError: 134479:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2366.726389] LNet: Removed LNI 192.168.203.139@tcp [ 2367.113590] Key type .llcrypt unregistered [ 2367.115229] Key type ._llcrypt unregistered [ 2378.017959] Key type ._llcrypt registered [ 2378.019278] Key type .llcrypt registered [ 2378.068150] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2378.520546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2378.529318] alg: No test for adler32 (adler32-zlib) [ 2379.427686] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2379.542645] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2381.143153] Key type lgssc registered [ 2381.643470] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2384.965969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2384.977062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2386.210272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2388.004831] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2391.849967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2392.021382] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2393.717053] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2396.645456] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2398.747909] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2401.485452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2401.647052] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2404.030836] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2407.072973] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2411.878397] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2422.205310] Lustre: *** cfs_fail_loc=159, val=0*** [ 2437.639097] Lustre: lustre-MDT0000: Client 18a48098-8db8-4101-9a18-e67e0f40adb6 (at 192.168.203.39@tcp) reconnecting [ 2438.636074] Lustre: *** cfs_fail_loc=159, val=0*** [ 2438.638478] Lustre: Skipped 1 previous similar message [ 2455.043396] Lustre: lustre-MDT0001: Client 18a48098-8db8-4101-9a18-e67e0f40adb6 (at 192.168.203.39@tcp) reconnecting [ 2455.691899] Lustre: *** cfs_fail_loc=159, val=0*** [ 2455.694597] Lustre: Skipped 2 previous similar messages [ 2471.939633] Lustre: lustre-MDT0001: Client 18a48098-8db8-4101-9a18-e67e0f40adb6 (at 192.168.203.39@tcp) reconnecting [ 2473.513981] Lustre: 138922:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2474.756284] Lustre: *** cfs_fail_loc=159, val=0*** [ 2474.759944] Lustre: Skipped 2 previous similar messages [ 2490.369559] Lustre: lustre-MDT0000: Client 524a043b-3c05-43a6-9d33-f0d81c244760 (at 192.168.203.39@tcp) reconnecting [ 2491.039499] Lustre: *** cfs_fail_loc=159, val=0*** [ 2491.042238] Lustre: Skipped 11 previous similar messages [ 2506.750161] Lustre: lustre-MDT0000: Client 524a043b-3c05-43a6-9d33-f0d81c244760 (at 192.168.203.39@tcp) reconnecting [ 2507.293562] Lustre: 139215:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2509.282479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2509.283093] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2509.283719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2509.287209] Lustre: Skipped 1 previous similar message [ 2514.401621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2514.404168] Lustre: Skipped 2 previous similar messages [ 2514.715483] Lustre: server umount lustre-OST0000 complete [ 2519.520398] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2519.521248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2519.524398] Lustre: Skipped 1 previous similar message [ 2519.529304] Lustre: Skipped 1 previous similar message [ 2522.326762] Lustre: server umount lustre-MDT0000 complete [ 2523.616312] LustreError: 135851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126364 with bad export cookie 5095547438044069173 [ 2523.617679] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2523.620897] LustreError: 135851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2523.752228] Lustre: server umount lustre-MDT0001 complete [ 2527.552356] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2528.654985] Key type lgssc unregistered [ 2528.799625] LNet: 140263:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2528.803766] LNetError: 140263:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2528.812437] LNet: Removed LNI 192.168.203.139@tcp [ 2529.179660] Key type .llcrypt unregistered [ 2529.181921] Key type ._llcrypt unregistered [ 2539.278997] Key type ._llcrypt registered [ 2539.280442] Key type .llcrypt registered [ 2539.322122] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2539.751898] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2539.764253] alg: No test for adler32 (adler32-zlib) [ 2540.638978] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2540.745364] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2542.335260] Key type lgssc registered [ 2542.778330] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2545.802782] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2545.812655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2546.992155] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2548.469953] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2551.680068] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2551.842153] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2553.525772] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2556.575153] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2558.758777] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2561.476510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2561.633068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2562.665569] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2563.846578] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2566.390566] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2572.902899] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2580.108250] Lustre: 144258:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2580.761428] Lustre: 144306:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2581.259018] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2582.490943] Lustre: 144454:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2582.857385] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2582.860704] Lustre: Skipped 1 previous similar message [ 2583.206402] Lustre: 144553:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2586.324409] Lustre: server umount lustre-OST0000 complete [ 2588.640926] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2588.641701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2588.647852] Lustre: Skipped 1 previous similar message [ 2593.760687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2593.763690] Lustre: Skipped 2 previous similar messages [ 2597.881673] Lustre: server umount lustre-MDT0000 complete [ 2598.880431] LustreError: 141652:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2598.889160] LustreError: 141652:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2599.303744] LustreError: 141633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126440 with bad export cookie 669367584905887179 [ 2599.306224] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2599.308237] LustreError: 141633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2609.690806] Lustre: server umount lustre-MDT0001 complete [ 2613.750651] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2614.959485] Key type lgssc unregistered [ 2615.102498] LNet: 145611:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2615.106338] LNetError: 145611:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2615.116622] LNet: Removed LNI 192.168.203.139@tcp [ 2615.464972] Key type .llcrypt unregistered [ 2615.466616] Key type ._llcrypt unregistered [ 2625.593308] Key type ._llcrypt registered [ 2625.595018] Key type .llcrypt registered [ 2625.639255] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2626.011378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2626.032030] alg: No test for adler32 (adler32-zlib) [ 2626.910665] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2627.017342] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2628.608102] Key type lgssc registered [ 2629.056921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2632.026150] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2632.034264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2633.217128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2634.660507] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2637.804470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2637.952571] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2639.453932] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2642.213194] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2644.378042] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2647.111299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2647.275883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2648.296261] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2649.814800] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2652.911074] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2659.562730] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2665.712917] Lustre: *** cfs_fail_loc=159, val=0*** [ 2665.717638] Lustre: Skipped 1 previous similar message [ 2681.357556] Lustre: lustre-MDT0000: Client 28d45556-35d2-4436-9ab6-5c4b686fe554 (at 192.168.203.39@tcp) reconnecting [ 2684.384511] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2684.387629] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2684.393915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2685.410307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2685.411294] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2685.412172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2688.786459] Lustre: server umount lustre-OST0000 complete [ 2690.528234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2690.529083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2690.532045] Lustre: Skipped 1 previous similar message [ 2690.533723] Lustre: Skipped 1 previous similar message [ 2695.648511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2695.650875] Lustre: Skipped 2 previous similar messages [ 2700.768074] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2700.770601] Lustre: Skipped 1 previous similar message [ 2704.351116] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2704.408277] Lustre: server umount lustre-MDT0000 complete [ 2705.681418] LustreError: 146986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126546 with bad export cookie 15417459115506387794 [ 2705.684506] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2705.686226] LustreError: 146986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2705.805418] Lustre: server umount lustre-MDT0001 complete [ 2709.695225] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2710.796603] Key type lgssc unregistered [ 2710.928448] LNet: 150682:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2710.931969] LNetError: 150682:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2710.940296] LNet: Removed LNI 192.168.203.139@tcp [ 2711.278709] Key type .llcrypt unregistered [ 2711.280423] Key type ._llcrypt unregistered [ 2721.278695] Key type ._llcrypt registered [ 2721.280327] Key type .llcrypt registered [ 2721.325331] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2721.651357] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2721.665324] alg: No test for adler32 (adler32-zlib) [ 2722.534278] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2722.644114] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2724.239147] Key type lgssc registered [ 2724.664738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2727.446176] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2727.453620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2728.603614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2730.003524] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2732.874194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2733.035070] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2734.552460] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2737.195112] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2739.157577] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2741.650586] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2741.799527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2742.821909] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2743.793077] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2746.273938] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2754.087064] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2769.302984] Lustre: lustre-OST0000: evicting ff92857a-a1b3-4a4f-a304-792edfda913f (at 192.168.203.39@tcp) by administrative request [ 2769.307716] Lustre: Skipped 1 previous similar message [ 2793.952309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2793.955976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2793.961254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2794.976580] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2794.977358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2794.977506] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2794.989996] Lustre: Skipped 1 previous similar message [ 2799.571349] Lustre: server umount lustre-OST0000 complete [ 2804.704765] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2804.708408] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2804.713845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2807.001084] Lustre: server umount lustre-MDT0000 complete [ 2808.381176] LustreError: 152055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126649 with bad export cookie 5601838750513568424 [ 2808.385101] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2808.387181] LustreError: 152055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2808.498741] Lustre: server umount lustre-MDT0001 complete [ 2812.410573] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2813.458344] Key type lgssc unregistered [ 2813.590569] LNet: 156166:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2813.594414] LNetError: 156166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2813.604491] LNet: Removed LNI 192.168.203.139@tcp [ 2813.945619] Key type .llcrypt unregistered [ 2813.946691] Key type ._llcrypt unregistered [ 2839.327519] Key type ._llcrypt registered [ 2839.329045] Key type .llcrypt registered [ 2839.370288] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2839.713587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2839.724989] alg: No test for adler32 (adler32-zlib) [ 2840.592877] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2840.683566] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2842.271167] Key type lgssc registered [ 2842.679429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2845.491225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2845.498634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2846.652167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2848.019570] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2850.927772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2851.062793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2852.450072] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2854.845249] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2856.682198] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2858.976500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2859.136881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2860.198574] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2861.147742] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2863.584238] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2871.466444] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2880.991874] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2880.995330] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2881.001126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2882.018188] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2882.018783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2882.019127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2883.154620] Lustre: server umount lustre-OST0000 complete [ 2887.138033] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2887.138222] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2887.139030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2887.139036] Lustre: Skipped 1 previous similar message [ 2887.143607] Lustre: Skipped 1 previous similar message [ 2890.650767] Lustre: server umount lustre-MDT0000 complete [ 2891.942136] LustreError: 158606:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126732 with bad export cookie 5053204169496763963 [ 2891.945982] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2891.947206] LustreError: 158606:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2892.072181] Lustre: server umount lustre-MDT0001 complete [ 2895.820368] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2896.937677] Key type lgssc unregistered [ 2897.084594] LNet: 162183:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2897.088214] LNetError: 162183:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2897.098334] LNet: Removed LNI 192.168.203.139@tcp [ 2897.417586] Key type .llcrypt unregistered [ 2897.419631] Key type ._llcrypt unregistered [ 2906.600121] Key type ._llcrypt registered [ 2906.601762] Key type .llcrypt registered [ 2906.641375] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2907.057139] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2907.079134] alg: No test for adler32 (adler32-zlib) [ 2907.938021] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 2908.032350] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 2909.615141] Key type lgssc registered [ 2909.984120] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2915.266563] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:45:55 (1784126755) [ 2920.830363] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 2924.393714] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 2936.779397] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2936.789353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2937.874121] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2937.887772] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2937.923923] Lustre: lustre-MDT0000: new disk, initializing [ 2937.948466] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2937.955477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2939.392892] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2943.597933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2943.648449] Lustre: 166417:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2943.675025] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2943.676923] Lustre: Skipped 1 previous similar message [ 2943.717783] Lustre: lustre-MDT0001: new disk, initializing [ 2943.756449] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2943.769779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2943.773431] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2945.137703] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2948.875091] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2950.866970] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2953.382034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2953.498112] Lustre: lustre-OST0000: new disk, initializing [ 2953.500229] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2953.503807] Lustre: 168385:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2953.536825] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2954.039285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2954.044680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2954.079877] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2955.734866] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 2959.631074] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2964.448667] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2964.450866] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2964.451567] LustreError: Skipped 1 previous similar message [ 2964.457771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2964.461251] Lustre: Skipped 1 previous similar message [ 2969.572724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2969.575335] Lustre: Skipped 1 previous similar message [ 2974.692287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2974.695174] Lustre: Skipped 1 previous similar message [ 2977.247150] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2977.299667] Lustre: server umount lustre-OST0000 complete [ 2979.808387] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2979.809300] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2979.815769] Lustre: Skipped 1 previous similar message [ 2979.818360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2979.820659] Lustre: Skipped 1 previous similar message [ 2984.669650] Lustre: server umount lustre-MDT0000 complete [ 2985.856338] LustreError: 166408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784126826 with bad export cookie 17779548268332458269 [ 2985.858779] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2985.860336] LustreError: 166408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2985.972812] Lustre: server umount lustre-MDT0001 complete [ 2989.386522] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 2990.314543] Key type lgssc unregistered [ 2990.445318] LNet: 170286:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2990.448204] LNetError: 170286:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2990.462942] LNet: Removed LNI 192.168.203.139@tcp [ 2990.745949] Key type .llcrypt unregistered [ 2990.747024] Key type ._llcrypt unregistered [ 2999.714753] Key type ._llcrypt registered [ 2999.715605] Key type .llcrypt registered [ 2999.752374] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3000.177468] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3000.198398] alg: No test for adler32 (adler32-zlib) [ 3001.050366] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 3001.132534] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 3002.711145] Key type lgssc registered [ 3003.068668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3005.609755] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3005.615951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3006.762200] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3008.136193] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3010.968890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3011.094711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3012.473766] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3014.806367] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3016.584312] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3018.815351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3018.923766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3020.718196] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3022.920564] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3032.229748] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3035.892253] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3035.900839] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3035.905578] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3035.909109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.913190] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3036.251415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.615476] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3036.619455] Lustre: Skipped 1 previous similar message [ 3036.628548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3036.634270] Lustre: Skipped 1 previous similar message [ 3036.636154] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3036.638918] Lustre: Skipped 1 previous similar message [ 3036.641395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.646455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3036.648808] Lustre: Skipped 1 previous similar message [ 3036.956858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.268932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.568537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.869521] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3037.873464] Lustre: Skipped 3 previous similar messages [ 3037.879722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3037.883943] Lustre: Skipped 3 previous similar messages [ 3037.886137] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3037.888919] Lustre: Skipped 3 previous similar messages [ 3037.890890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.895274] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3037.897456] Lustre: Skipped 3 previous similar messages [ 3038.202342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.519305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.824199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.158475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.443974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.743565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.041840] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3040.045669] Lustre: Skipped 6 previous similar messages [ 3040.052688] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3040.057613] Lustre: Skipped 6 previous similar messages [ 3040.059800] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3040.063110] Lustre: Skipped 6 previous similar messages [ 3040.068910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.074534] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3040.077452] Lustre: Skipped 6 previous similar messages [ 3040.375920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.693206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.024430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.337547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.649325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.956270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.268292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.580199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.918814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.233166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.550390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.864600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.171401] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3044.174653] Lustre: Skipped 12 previous similar messages [ 3044.181274] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3044.187172] Lustre: Skipped 12 previous similar messages [ 3044.189246] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3044.192233] Lustre: Skipped 12 previous similar messages [ 3044.193923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.198271] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3044.200576] Lustre: Skipped 12 previous similar messages [ 3044.488936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.778169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.073503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.393380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.679558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.969938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.258411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.556414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.859297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.167037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.464050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.773816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.079441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.389387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.681172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.981330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.275804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.557699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.841865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.140288] LustreError: 171695:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3050.144232] LustreError: 171695:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3050.144831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.443517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.748375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.071675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.419944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.732475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.046553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.344538] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3052.347529] Lustre: Skipped 26 previous similar messages [ 3052.353613] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3052.358241] Lustre: Skipped 26 previous similar messages [ 3052.360255] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3052.362792] Lustre: Skipped 26 previous similar messages [ 3052.364767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.370549] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3052.372841] Lustre: Skipped 26 previous similar messages [ 3052.686552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.995229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.300720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.604579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.923511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.238475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.575633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.878778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.211774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.532669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.840579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.158780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.494222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.822589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.159724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.461751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.779271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.077386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.378043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.672937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.993108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.291364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.589395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.886098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.200086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.522716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.857084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.186569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.520071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.870474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.212035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.531794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.858937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.165086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.470788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.773719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.117628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.432408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.740538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.041873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.367753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.701963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.017969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.342932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.676706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.025609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.381415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.706315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.042946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.370668] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3068.373013] Lustre: Skipped 49 previous similar messages [ 3068.380178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3068.385338] Lustre: Skipped 49 previous similar messages [ 3068.387597] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3068.391323] Lustre: Skipped 49 previous similar messages [ 3068.394270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.399668] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3068.402473] Lustre: Skipped 49 previous similar messages [ 3068.712160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.042695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.362348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.674727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.984450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.305910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.631684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.953658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.270610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.602757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.924693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.277509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.629340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.958557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.277748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.603021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.929255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.245484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.580403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.917838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.240616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.590937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.909216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.253142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.572671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.933109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.297336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.637375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.968714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.281353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.604576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.925692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.252585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.580055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.902643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.249197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.605075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.951054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.274293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.597492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.914566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.256479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.588227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.896930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.189880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.499586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.811208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.119597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.429102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.758597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.057157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.357459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.664554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.978246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.287412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.586184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.929822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.244232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.548549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.873361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.181517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.499270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.801891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.093993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.388296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.685873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.978553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.276349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.580073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.888898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.181935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.472303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.780393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.121413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.416380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.707845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.013916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.327770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.641828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.950688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.262030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.594049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.909502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.215210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.513201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.831398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.145345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.433364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.727987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.027674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.318485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.602986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.880521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.188317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.492990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.797678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.123485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.445985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.758588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.082116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.397658] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3100.400276] Lustre: Skipped 100 previous similar messages [ 3100.406152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3100.411074] Lustre: Skipped 100 previous similar messages [ 3100.413085] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3100.418276] Lustre: Skipped 100 previous similar messages [ 3100.420783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.424769] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3100.427912] Lustre: Skipped 100 previous similar messages [ 3100.749111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.083885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.385882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.693409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.015387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.345033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.646353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.953362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.284926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.607609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.943140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.258549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.585863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.909608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.235855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.560364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.903606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.220952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.567608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.896681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.249579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.581758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.904554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.234721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.570911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.893172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.210428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.517475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.845173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.166177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.483645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.810599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.156553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.485411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.795508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.113394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.435813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.757620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.052267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.373952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.689921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.006136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.321492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.639612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.963177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.282679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.596848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.922791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.260120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.575714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.889229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.220350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.551616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.873826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.187871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.522466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.860428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.197041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.542757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.877083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.225715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.575624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.914676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.248427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.597492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.950852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.287353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.616226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.931079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.258148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.568323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.890078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.206219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.513902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.832170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.176344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.497184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.809812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.128258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.444159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.752312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.072056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.385465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.710796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.023295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.349621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.659171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.992990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.302711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.622569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.938606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.251804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.562965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.872454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.195253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.520169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.833777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.149296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.465238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.797046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.120270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.434501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.759086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.083987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.393693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.729264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.049247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.390921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.724756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.045213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.354928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.678799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.980812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.291715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.626917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.972484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.280818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.600517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.934761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.277254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.617750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.942254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.288059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.622219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.941165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.261624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.597497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.936930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.259184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.581319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.901875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.250726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.569297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.886117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.200142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.537929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.860073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.196943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.526141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.855519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.165213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.471182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.778098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.097732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.403202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.720936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.026061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.355729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.665869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.969414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.276078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.619419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.926429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.239061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.558140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.907421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.244195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.579642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.899473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.234168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.553838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.863958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.169246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.480643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.789770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.112872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.433557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.754378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.077533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.412892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.724087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.054717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.374661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.681831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.994583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.323361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.630657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.930341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.232708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.558207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.853400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.154227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.466792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.783103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.093983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.411500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.714959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.039942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.368784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.697332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.008943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.335862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.646308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.961456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.286813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.622727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.935555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.242245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.547173] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3164.549579] Lustre: Skipped 198 previous similar messages [ 3164.555783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3164.560042] Lustre: Skipped 198 previous similar messages [ 3164.562148] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3164.564544] Lustre: Skipped 198 previous similar messages [ 3164.566412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.570841] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3164.573824] Lustre: Skipped 198 previous similar messages [ 3164.883662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.188612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.498199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.801194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.123402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.439744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.777449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.110482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.452149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.807828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.135612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.488302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.846122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.190991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.518721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.848976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.162736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.472910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.787418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.094309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.390465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.695442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.025828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.322195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.627833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.959068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.305127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.619951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.930669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.242250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.549530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.844985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.145985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.441035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.745562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.042804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.369733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.674834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.012904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.314505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.630123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.917162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.221203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.510802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.811070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.115406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.449881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.729823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.020238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.323117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.644793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.968593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.273449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.576611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.885422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.202023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.548020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.865839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.205331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.517526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.822763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.118572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.438587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.767932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.080854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.398932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.735526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.031148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.328972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.630087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.951723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.254637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.551955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.876131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.203344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.513876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.834483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.150652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.481925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.810608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.145755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.468340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.794960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.109929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.416061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.726502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.063791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.384739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.693180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.007983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.342441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.657122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.999268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.313150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.641087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.979813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.318161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.630357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.953424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.262455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.588444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.914506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.247232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.578485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.901181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.217032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.556171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.880059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.199807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.519890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.867878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.217233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.553893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.873753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.199066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.525600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.859287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.171555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.497377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.812662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.145915] LustreError: 171695:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3203.149742] LustreError: 171695:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3203.153631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.490286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.811652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.132261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.458864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.755119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.063900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.373635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.701146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.015705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.336105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.664437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.013382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.324013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.635662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.942707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.266599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.574387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.872855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.184170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.500749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.819227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.128892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.444477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.771376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.079225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.401589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.715402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.036339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.339791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.639021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.940273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.269209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.579092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.886485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.210855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.548260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.868754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.186400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.505880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.846975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.174766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.501268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.826639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.163238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.500395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.842426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.168632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.508479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.835278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.159084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.488746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.827116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.151541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.479210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.791258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.135665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.452449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.768235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.108831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.440086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.756181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.083986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.391969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.707467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.007540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.335623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.657267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.992335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.327742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.656204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.977195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.334388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.656105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.977790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.309361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.665675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.004822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.339868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.650031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.972525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.277893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.589990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.897362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.225813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.531632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.873944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.228078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.588860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.924383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.256023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.602191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.939547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.254570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.565977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.905181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.240717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.558116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.889623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.204252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.535919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.852220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.161577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.479918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.820863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.142934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.491699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.825860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.158859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.492754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.808014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.119170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.444887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.769589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.106274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.423660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.776389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.085297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.406348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.741204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.079981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.410167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.745845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.066381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.408480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.736856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.079485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.410235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.751859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.078113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.412143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.746077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.084147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.422389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.763728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.094754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.450453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.791067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.117421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.459566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.836041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.190480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.526458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.870345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.219466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.576546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.917093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.268235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.650265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.004918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.367429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.711532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.038982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.369694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.727270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.058716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.384146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.712869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.069750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.403578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.746387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.067697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.407159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.769762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.116507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.443831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.790408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.111971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.446172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.812080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.165148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.522239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.899398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.264322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.623360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.960112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.292645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.622183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.985360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.344085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.666706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.993197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.321311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.653600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.994217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.323705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.683432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.033493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.357808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.676383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.001517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.329900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.655800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.977946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.313692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.658884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.988742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.317492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.638841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.975423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.307563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.627654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.959677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.290470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.608165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.934487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.277701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.612930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.946052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.295023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.641465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.969806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.288479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.618087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.959335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.288186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.629940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.967769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.324381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.653204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.979099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.305777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.663044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.007364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.365227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.702365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.039446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.371231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.693394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.040752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.389956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.715587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.057761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.394042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.744762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.071589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.409352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.742888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.074535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.415474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.747590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.097252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.446200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.783173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.114869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.458070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.797697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.133539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.465061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.797623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.121756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.431174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.762754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.075731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.404389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.735522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.100793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.437705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.758761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.078775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.408032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.735932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.054576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.374373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.719413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.044772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.370608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.689688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.022626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.340188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.649592] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3292.651982] Lustre: Skipped 390 previous similar messages [ 3292.657827] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3292.663187] Lustre: Skipped 390 previous similar messages [ 3292.665762] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3292.668979] Lustre: Skipped 390 previous similar messages [ 3292.670892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.675634] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3292.678064] Lustre: Skipped 390 previous similar messages [ 3292.985182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.323963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.651288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.981625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.295949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.649781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.991640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.310631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.635281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.963217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.285851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.618245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.933745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.263101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.591898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.955205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.287468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.617046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.933746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.248157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.567048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.900863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.227301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.557562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.881195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.210230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.527263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.844360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.159730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.466666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.788769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.133309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.443250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.742502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.053753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.399288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.736451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.078493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.433320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.777520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.083319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.386610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.701900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.026148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.336249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.663900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.986309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.317750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.632644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.945460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.254959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.575127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.887650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.209684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.512156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.831011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.125907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.429611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.739292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.077457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.389122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.692422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.012657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.358294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.679691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.012255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.347411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.693155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.020763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.349809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.673657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.015413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.312940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.626569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.951676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.260773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.558715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.867548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.169135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.494668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.835623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.150572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.468196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.800942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.129404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.453684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.747747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.063062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.363304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.658122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.942530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.256061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.584046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.885548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.187181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.532757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.848598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.149924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.462753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.806975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.138126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.460220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.815279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.155588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.464913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.784586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.091270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.415883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.739588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.032912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.323448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.631679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.939909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.263069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.583479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.912906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.228856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.541870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.841530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.158708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.485374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.783737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.096723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.437256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.739015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.045981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.355756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.676068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.968357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.267595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.569180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.891247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.187700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.501880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.832938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.175772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.476772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.778310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.103830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.433464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.748788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.057335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.400207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.734803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.056462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.401908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.714337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.051226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.376595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.713515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.014404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.337933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.659762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.954780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.259031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.595733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.935224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.241069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.540991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.896891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.244276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.557282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.893460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.237866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.547055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.876361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.213727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.549602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.862546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.173597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.479867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.811228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.129987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.445916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.770797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.151759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.490796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.800351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.119309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.457572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.764189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.092906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.410719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.742381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.047081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.385306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.706894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.041876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.362390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.679939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.006336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.333988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.663703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.978820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.297979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.669176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.004921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.335766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.651513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.978670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.270510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.556459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.866872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.191904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.489306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.322397] LustreError: 173493:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3440.325260] LustreError: 171695:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3512.801182] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3512.802584] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3512.804931] LustreError: Skipped 1 previous similar message [ 3513.825985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3513.828144] Lustre: Skipped 2 previous similar messages [ 3515.767553] Lustre: server umount lustre-OST0000 complete [ 3517.919789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3517.922871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3519.188740] Lustre: server umount lustre-MDT0000 complete [ 3520.464086] LustreError: 225750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127361 with bad export cookie 11023647981819102548 [ 3520.466611] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3520.468710] LustreError: 225750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3526.639853] Lustre: server umount lustre-MDT0001 complete [ 3530.261527] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 3531.279778] Key type lgssc unregistered [ 3531.406303] LNet: 226703:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3531.408811] LNetError: 226703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3531.416283] LNet: Removed LNI 192.168.203.139@tcp [ 3531.722823] Key type .llcrypt unregistered [ 3531.723866] Key type ._llcrypt unregistered [ 3540.621642] Key type ._llcrypt registered [ 3540.622965] Key type .llcrypt registered [ 3540.657174] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3540.937958] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3540.946040] alg: No test for adler32 (adler32-zlib) [ 3541.795101] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 3541.882792] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 3543.463163] Key type lgssc registered [ 3543.795123] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3546.280059] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3546.287783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3547.441371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3548.794227] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3551.616409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3551.743152] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3553.079673] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3555.494515] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3557.342241] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3559.522865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3559.635109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3561.444196] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3563.751423] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3572.964212] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3576.596876] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3577.195603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3577.200053] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3577.203112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3577.207351] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3577.207369] LustreError: 228113:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3578.337588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3578.339100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3578.341584] Lustre: Skipped 1 previous similar message [ 3579.379563] Lustre: lustre-OST0000: Not available for connect from 192.168.203.39@tcp (stopping) [ 3579.382253] Lustre: Skipped 1 previous similar message [ 3579.925240] Lustre: server umount lustre-OST0000 complete [ 3597.287120] LustreError: 228113:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3607.825627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3607.921588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3609.727791] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3612.030704] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3618.148192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.152646] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.152750] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3618.158108] Lustre: Skipped 1 previous similar message [ 3620.698250] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 8 sec [ 3622.507164] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff983e078c6800.ost_server_uuid 50 [ 3622.984231] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff983e078c6800.ost_server_uuid in FULL state after 0 sec [ 3628.514765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3628.522263] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3628.522351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3628.528927] Lustre: Skipped 1 previous similar message [ 3630.030316] Lustre: server umount lustre-OST0000 complete [ 3633.632778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3633.632885] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3633.633538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3633.642618] Lustre: Skipped 2 previous similar messages [ 3637.396398] Lustre: server umount lustre-MDT0000 complete [ 3638.645158] LustreError: 230841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127479 with bad export cookie 4288497116906993890 [ 3638.648669] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3638.649164] LustreError: 230841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3638.760522] Lustre: server umount lustre-MDT0001 complete [ 3642.430090] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 3643.472286] Key type lgssc unregistered [ 3643.606340] LNet: 233317:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3643.608552] LNetError: 233317:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3643.615295] LNet: Removed LNI 192.168.203.139@tcp [ 3643.922376] Key type .llcrypt unregistered [ 3643.923385] Key type ._llcrypt unregistered [ 3655.440881] Key type ._llcrypt registered [ 3655.442242] Key type .llcrypt registered [ 3655.486990] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3655.815662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3655.837063] alg: No test for adler32 (adler32-zlib) [ 3656.690977] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 3656.784326] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 3658.367164] Key type lgssc registered [ 3658.751234] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3661.817907] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3661.824684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3662.975885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3664.332240] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3667.240737] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3667.374871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3668.730979] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3669.720563] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3672.021691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3672.145117] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3673.193637] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3674.427926] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3677.176771] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3677.240104] Lustre: lustre-OST0001: new disk, initializing [ 3677.242412] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3677.245194] Lustre: 237329:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3677.276072] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3679.185632] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3682.306803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3682.310521] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3682.340867] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3684.518075] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3688.035645] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3697.634237] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3697.635621] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3697.638572] Lustre: Skipped 3 previous similar messages [ 3697.643850] Lustre: Skipped 3 previous similar messages [ 3700.512260] Lustre: server umount lustre-MDT0000 complete [ 3701.718129] LustreError: 238098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127542 with bad export cookie 6477859633616626975 [ 3701.720858] LustreError: 235756:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 3701.722455] LustreError: 238098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3701.724387] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3701.847199] Lustre: server umount lustre-MDT0001 complete [ 3713.415348] Lustre: server umount lustre-OST0000 complete [ 3724.958608] Lustre: server umount lustre-OST0001 complete [ 3731.944483] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3735.530186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3735.674689] LustreError: 240438:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3735.683835] LustreError: 240438:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3735.694818] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3737.038649] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3739.931236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3741.387278] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3742.340728] Lustre: 241579:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3744.351679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3744.463072] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3744.464875] Lustre: Skipped 1 previous similar message [ 3745.507876] LustreError: 242126:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3745.510783] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 3746.210920] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3748.953860] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3750.058305] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 3750.863799] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3753.913903] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3755.001660] Lustre: 243448:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3760.611278] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3760.614553] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3760.614862] Lustre: Skipped 2 previous similar messages [ 3760.614960] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3760.616954] Lustre: Skipped 1 previous similar message [ 3765.401088] Lustre: server umount mylustre-MDT0000 complete [ 3765.729331] LustreError: 240438:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3765.734518] LustreError: 240438:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3766.614163] LustreError: 243648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127607 with bad export cookie 6477859633616629824 [ 3766.617748] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3766.618608] LustreError: 243648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3766.740239] Lustre: server umount mylustre-MDT0001 complete [ 3778.058926] Lustre: server umount mylustre-OST0000 complete [ 3789.531669] Lustre: server umount mylustre-OST0001 complete [ 3796.021428] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3799.255899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3799.415867] LustreError: 245752:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3799.435611] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3799.437751] Lustre: Skipped 1 previous similar message [ 3800.735787] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3803.433117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3804.791107] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3805.720493] Lustre: 246896:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3807.848147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3809.841436] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3810.021559] LustreError: 247249:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3810.026037] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 3812.671891] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3813.802173] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 3814.610575] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3817.634966] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3818.774674] Lustre: 248763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3824.098505] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3824.098812] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3824.101709] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3824.101715] Lustre: Skipped 2 previous similar messages [ 3824.112487] Lustre: Skipped 2 previous similar messages [ 3826.839799] Lustre: server umount tfs-MDT0000 complete [ 3828.019142] LustreError: 245729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127668 with bad export cookie 6477859633616631777 [ 3828.022828] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3828.023599] LustreError: 245729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3828.138283] Lustre: server umount tfs-MDT0001 complete [ 3839.617505] Lustre: server umount tfs-OST0000 complete [ 3851.106504] Lustre: server umount tfs-OST0001 complete [ 3857.781277] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3860.970150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3861.121706] LustreError: 251021:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3861.139300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3861.141615] Lustre: Skipped 3 previous similar messages [ 3862.464995] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3865.100883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3866.530845] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3867.467180] Lustre: 252162:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3869.681970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3871.676015] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3871.847038] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 3874.421337] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3875.563851] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 3876.391918] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3879.434413] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3880.581770] Lustre: 254029:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3882.759948] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:02:03 (1784127723) [ 3896.288727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3896.291530] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3896.293093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3896.295158] Lustre: Skipped 5 previous similar messages [ 3896.296683] Lustre: Skipped 5 previous similar messages [ 3900.694054] Lustre: server umount lustre-MDT0000 complete [ 3901.408320] LustreError: 251022:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3901.413933] LustreError: 251022:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3901.903181] LustreError: 254031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127742 with bad export cookie 6477859633616633464 [ 3901.906913] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3906.528894] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3906.529640] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3906.533093] Lustre: Skipped 1 previous similar message [ 3916.255113] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3916.335028] Lustre: server umount lustre-MDT0001 complete [ 3923.710887] Lustre: server umount lustre-OST0000 complete [ 3939.295118] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3939.349852] Lustre: server umount lustre-OST0001 complete [ 3941.529411] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 3944.786381] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 3956.709962] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3956.709962] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3956.709962] [ 3956.718154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3956.806353] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3956.818184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3956.853164] Lustre: lustre-MDT0000: new disk, initializing [ 3956.872613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3956.874758] Lustre: Skipped 3 previous similar messages [ 3956.879354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3958.257263] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3962.437530] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3962.437530] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3962.437530] [ 3962.460745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3962.501125] Lustre: 257209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3962.529040] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3962.530719] Lustre: Skipped 1 previous similar message [ 3962.564408] Lustre: lustre-MDT0001: new disk, initializing [ 3962.611557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3962.616177] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3963.968990] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3968.128631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3968.235474] Lustre: lustre-OST0000: new disk, initializing [ 3968.237338] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3968.239908] Lustre: 258794:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3969.398554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3969.401925] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3969.438096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3970.316550] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3974.472064] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3974.539380] Lustre: lustre-OST0001: new disk, initializing [ 3974.541773] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3974.545859] Lustre: 259663:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3975.753675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3975.757337] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3975.793221] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3976.550708] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 3980.772797] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:03:41 (1784127821) [ 3984.229869] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:03:44 (1784127824) [ 3996.644492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3996.645513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3996.648719] Lustre: Skipped 1 previous similar message [ 3996.652749] Lustre: Skipped 4 previous similar messages [ 4001.721877] Lustre: server umount lustre-OST0000 complete [ 4001.764025] LustreError: 258779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4001.770222] LustreError: 258779:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 4003.807939] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4009.114402] Lustre: server umount lustre-MDT0000 complete [ 4010.382195] LustreError: 257204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127851 with bad export cookie 6477859633616635018 [ 4010.385763] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4010.387279] LustreError: 257204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4016.032034] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4016.034149] Lustre: Skipped 5 previous similar messages [ 4016.568053] Lustre: server umount lustre-MDT0001 complete [ 4020.227301] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4030.048349] Lustre: server umount lustre-OST0001 complete [ 4032.120398] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 4038.386129] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4050.373601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4050.473420] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4050.485582] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4050.523606] Lustre: lustre-MDT0000: new disk, initializing [ 4050.550018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4051.960072] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4056.108362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4056.152207] Lustre: 264186:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4056.258109] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4057.650632] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4061.371465] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4063.231853] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4065.445302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4065.555362] Lustre: 266156:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4067.321974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4067.325419] Lustre: Skipped 1 previous similar message [ 4067.328083] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4067.353890] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4067.516491] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4071.124592] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4077.539096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4077.539246] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.541713] LustreError: Skipped 1 previous similar message [ 4077.546632] Lustre: Skipped 4 previous similar messages [ 4077.549081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4077.552260] Lustre: Skipped 1 previous similar message [ 4079.505130] Lustre: server umount lustre-OST0000 complete [ 4086.866090] Lustre: server umount lustre-MDT0000 complete [ 4087.775874] LustreError: 264199:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4087.780530] LustreError: 264199:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4088.037136] LustreError: 264179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127928 with bad export cookie 6477859633616636488 [ 4088.040689] LustreError: 264179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4088.041110] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4088.149762] Lustre: server umount lustre-MDT0001 complete [ 4091.594121] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4092.632685] Key type lgssc unregistered [ 4092.766359] LNet: 267914:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4092.768952] LNetError: 267914:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4092.779368] LNet: Removed LNI 192.168.203.139@tcp [ 4093.084113] Key type .llcrypt unregistered [ 4093.085311] Key type ._llcrypt unregistered [ 4102.396190] Key type ._llcrypt registered [ 4102.397431] Key type .llcrypt registered [ 4102.431050] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4102.734935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4102.780214] alg: No test for adler32 (adler32-zlib) [ 4103.624787] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 4103.712691] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4105.287155] Key type lgssc registered [ 4105.619937] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4108.104803] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4108.111249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4109.248118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4110.510838] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4113.343333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4113.468450] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4114.766227] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4117.008616] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4118.684726] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4120.813383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4120.919416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4122.754141] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4125.133531] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4125.475953] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4127.011908] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4127.017390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4132.238537] Lustre: server umount lustre-OST0000 complete [ 4133.856021] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4133.858364] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4133.862640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4137.440569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4137.441298] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4139.606194] Lustre: server umount lustre-MDT0000 complete [ 4140.860221] LustreError: 269334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784127981 with bad export cookie 12060152547822049713 [ 4140.863617] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4140.863956] LustreError: 269334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4140.985305] Lustre: server umount lustre-MDT0001 complete [ 4144.577500] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4145.603324] Key type lgssc unregistered [ 4145.720418] LNet: 272932:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4145.723987] LNetError: 272932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4145.732294] LNet: Removed LNI 192.168.203.139@tcp [ 4146.036909] Key type .llcrypt unregistered [ 4146.038929] Key type ._llcrypt unregistered [ 4152.612319] Key type ._llcrypt registered [ 4152.613857] Key type .llcrypt registered [ 4152.653376] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4152.923339] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4152.931256] alg: No test for adler32 (adler32-zlib) [ 4153.782311] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 4153.871645] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4155.447160] Key type lgssc registered [ 4155.784225] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4158.605493] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4158.612939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4174.047452] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4179.167337] LustreError: 274104:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.139@tcp: failed processing log, type 4: rc = -110 [ 4205.791182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4207.604498] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4209.985519] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4210.675600] Lustre: server umount lustre-OST0000 complete [ 4216.007538] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4216.999270] Key type lgssc unregistered [ 4217.121338] LNet: 275731:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4217.124303] LNetError: 275731:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4217.137300] LNet: Removed LNI 192.168.203.139@tcp [ 4217.437134] Key type .llcrypt unregistered [ 4217.438209] Key type ._llcrypt unregistered [ 4230.686526] Key type ._llcrypt registered [ 4230.689118] Key type .llcrypt registered [ 4230.741406] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4231.107103] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4231.157463] alg: No test for adler32 (adler32-zlib) [ 4232.015228] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 4232.106696] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4233.687150] Key type lgssc registered [ 4234.068214] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4238.603587] loop0: detected capacity change from 0 to 204800000 [ 4238.664846] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4238.669319] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4238.799318] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.142953] loop0: detected capacity change from 0 to 204800000 [ 4239.207822] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.212163] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.337820] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.690031] loop0: detected capacity change from 0 to 204800000 [ 4239.752508] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.756634] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4239.796920] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4240.189983] loop0: detected capacity change from 0 to 204800000 [ 4240.248016] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4240.657958] loop0: detected capacity change from 0 to 204800000 [ 4240.669847] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4241.954354] loop0: detected capacity change from 0 to 204800000 [ 4241.963337] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4243.243550] loop0: detected capacity change from 0 to 204800000 [ 4243.252318] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4244.548293] loop0: detected capacity change from 0 to 204800000 [ 4244.559298] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4245.869992] loop0: detected capacity change from 0 to 204800000 [ 4245.897573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4245.907821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4245.941665] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4245.944760] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4245.949075] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4245.953707] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4245.957215] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4245.960467] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4245.964567] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4245.969058] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4245.972702] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4245.976738] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4245.981260] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4245.985342] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4245.989639] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4245.994074] Lustre: 279120:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4248.534825] Lustre: server umount MGS complete [ 4248.828748] loop0: detected capacity change from 0 to 204800000 [ 4248.858978] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4249.014715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4249.029865] Lustre: lustre-MDT0000: Aborting client recovery [ 4249.351041] loop1: detected capacity change from 0 to 204800000 [ 4249.380412] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4249.415826] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4249.420370] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4249.424285] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4249.428439] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4249.433185] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4249.437794] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4249.442138] Lustre: 279555:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4249.449552] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4249.514440] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4249.542423] Lustre: lustre-MDT0001: Aborting client recovery [ 4249.866265] loop2: detected capacity change from 0 to 204800000 [ 4249.898726] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4249.931812] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4249.936883] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4249.941315] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4249.945991] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4249.949945] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4249.954251] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4249.958099] Lustre: 279636:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4250.029828] Lustre: Found index 0 for lustre-OST0000, updating log [ 4250.042391] Lustre: 279655:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4250.068884] Lustre: lustre-OST0000: Aborting recovery [ 4250.399397] loop3: detected capacity change from 0 to 204800000 [ 4250.431359] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4250.463697] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4250.469246] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4250.475827] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4250.482849] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4250.488461] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4250.493748] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4250.499865] Lustre: 279721:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4250.532875] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4250.535888] Lustre: Skipped 1 previous similar message [ 4255.717914] LustreError: 279499:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4258.784782] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4258.785209] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4258.794138] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4258.797234] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4259.810273] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4259.810399] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4260.834378] LustreError: 279465:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 304us: rc = -2 [ 4275.167834] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4275.167846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4275.168248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4275.171823] Lustre: Skipped 2 previous similar messages [ 4278.303673] Lustre: server umount lustre-MDT0000 complete [ 4278.591635] LustreError: 279449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784128119 with bad export cookie 18289257168603506581 [ 4278.593229] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4278.596263] LustreError: 279449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4280.287632] LustreError: 279663:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4280.287717] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4280.288143] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4280.288149] Lustre: Skipped 3 previous similar messages [ 4280.292929] LustreError: 279663:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4280.297327] Lustre: Skipped 2 previous similar messages [ 4283.295622] LustreError: 279464:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4283.295649] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4283.301050] LustreError: 279464:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4283.303092] Lustre: Skipped 2 previous similar messages [ 4284.782087] Lustre: server umount lustre-MDT0001 complete [ 4291.202727] Lustre: server umount lustre-OST0000 complete [ 4297.623056] Lustre: server umount lustre-OST0001 complete [ 4331.727300] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:09:31 (1784128171) [ 4341.084587] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 4344.799662] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4364.148486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4364.280684] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4364.297980] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4364.350771] Lustre: lustre-MDT0000: new disk, initializing [ 4364.396364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4364.405176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4367.619866] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4377.230099] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4377.351421] Lustre: 283801:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4377.396304] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4377.400572] Lustre: Skipped 1 previous similar message [ 4377.486318] Lustre: lustre-MDT0001: new disk, initializing [ 4377.580169] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4377.634510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4377.657973] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4383.815600] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4400.607770] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4411.258528] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4421.491510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4421.790177] Lustre: lustre-OST0000: new disk, initializing [ 4421.793318] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4421.799041] Lustre: 285775:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4421.802128] Lustre: 285775:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 4421.875365] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4428.426396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4428.466082] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4428.707100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4430.937133] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4442.227184] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4446.627924] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4456.889596] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4463.357329] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4469.426252] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4515.813710] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4515.824915] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4515.835191] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4515.839214] Lustre: Skipped 1 previous similar message [ 4518.892356] Lustre: server umount lustre-OST0000 complete [ 4526.051606] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.053791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4526.067502] Lustre: Skipped 1 previous similar message [ 4526.070536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4526.074966] LustreError: Skipped 1 previous similar message [ 4526.084170] Lustre: Skipped 1 previous similar message [ 4527.749175] Lustre: server umount lustre-MDT0000 complete [ 4530.177416] LustreError: 283794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784128371 with bad export cookie 18289257168603509290 [ 4530.182419] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4530.184060] LustreError: 283794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4530.433112] Lustre: server umount lustre-MDT0001 complete [ 4536.205181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4539.411074] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4542.502315] Lustre: server umount MGS complete [ 4550.069413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.354481] LustreError: 289695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4550.375412] LustreError: 289708:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4550.422161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4552.959709] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4555.747267] LustreError: 289696:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4560.868744] LustreError: 289695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4561.058407] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4566.837482] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4578.976514] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4589.480128] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4598.545817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4598.786229] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4598.790907] Lustre: Skipped 1 previous similar message [ 4606.675296] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4615.171845] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4650.983791] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4650.986058] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4651.001650] Lustre: Skipped 1 previous similar message [ 4651.012141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4651.017176] Lustre: Skipped 1 previous similar message [ 4655.357841] Lustre: server umount lustre-OST0000 complete [ 4661.221456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.236680] Lustre: Skipped 1 previous similar message [ 4665.502714] Lustre: server umount lustre-MDT0000 complete [ 4666.337427] LustreError: 292425:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4666.366813] LustreError: 292425:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4669.771197] LustreError: 289675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784128510 with bad export cookie 18289257168603511229 [ 4669.785983] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4670.339624] Lustre: server umount lustre-MDT0001 complete [ 4682.092282] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 4684.981498] Key type lgssc unregistered [ 4685.214118] LNet: 293641:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4685.222374] LNetError: 293641:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4685.237680] LNet: Removed LNI 192.168.203.139@tcp [ 4685.970317] Key type .llcrypt unregistered [ 4685.973625] Key type ._llcrypt unregistered [ 4719.380419] Key type ._llcrypt registered [ 4719.383504] Key type .llcrypt registered [ 4719.528190] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 4734.561695] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 4735.829081] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4735.867557] alg: No test for adler32 (adler32-zlib) [ 4737.038892] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 4737.396763] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 4739.145141] Key type lgssc registered [ 4740.290697] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4770.013751] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4770.057408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4771.272818] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4771.320059] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4771.397802] Lustre: lustre-MDT0000: new disk, initializing [ 4771.469615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4771.487103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4775.171990] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4784.730504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4784.852137] Lustre: 297601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4784.933737] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4784.942219] Lustre: Skipped 1 previous similar message [ 4785.113910] Lustre: lustre-MDT0001: new disk, initializing [ 4785.261801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4785.298347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4785.313280] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4789.814171] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4799.232760] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4805.565687] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4811.964092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4812.239376] Lustre: lustre-OST0000: new disk, initializing [ 4812.242439] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4812.253910] Lustre: 299571:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4812.395810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4818.663301] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4820.551540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4820.561383] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4820.651922] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4827.317920] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4833.392913] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4845.860218] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4852.879317] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4857.179894] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4902.883191] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4902.883677] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4902.890310] LustreError: Skipped 1 previous similar message [ 4902.900558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4902.904473] Lustre: Skipped 1 previous similar message [ 4906.414627] Lustre: server umount lustre-OST0000 complete [ 4913.121146] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4913.123951] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4913.127248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4913.138635] Lustre: Skipped 2 previous similar messages [ 4918.240338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4918.246783] Lustre: Skipped 2 previous similar messages [ 4922.339388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4922.350725] Lustre: Skipped 1 previous similar message [ 4923.359129] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4923.455407] Lustre: server umount lustre-MDT0000 complete [ 4925.629362] LustreError: 300426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784128766 with bad export cookie 13184285961602733960 [ 4925.631333] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4925.637612] LustreError: 300426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4925.830819] Lustre: server umount lustre-MDT0001 complete [ 4930.490387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4932.997902] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4935.483855] Lustre: server umount MGS complete [ 4941.308638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4941.580200] LustreError: 303543:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4941.617350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4943.795854] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4946.913827] LustreError: 303531:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4946.928678] LustreError: 303531:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4948.351726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4948.561890] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4950.341130] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4953.591977] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4955.961907] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4959.033842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4959.189616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4961.864833] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 4964.960429] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4986.848109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4986.853466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4986.863221] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4988.898368] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4988.904579] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.212747] Lustre: server umount lustre-OST0000 complete [ 4994.019120] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.029875] Lustre: Skipped 1 previous similar message [ 4999.137547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4999.143187] Lustre: Skipped 5 previous similar messages [ 4999.307502] Lustre: server umount lustre-MDT0000 complete [ 5001.099173] LustreError: 304265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784128841 with bad export cookie 13184285961602735801 [ 5001.099748] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5001.107579] LustreError: 304265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5001.269825] Lustre: server umount lustre-MDT0001 complete [ 5006.942869] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 5008.547213] Key type lgssc unregistered [ 5008.710020] LNet: 307469:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5008.715721] LNetError: 307469:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5008.729689] LNet: Removed LNI 192.168.203.139@tcp [ 5009.170944] Key type .llcrypt unregistered [ 5009.173045] Key type ._llcrypt unregistered [ 5020.171537] Key type ._llcrypt registered [ 5020.173294] Key type .llcrypt registered [ 5020.240671] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 5027.406618] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5027.877053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5027.888584] alg: No test for adler32 (adler32-zlib) [ 5028.783790] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 5028.909749] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 5030.511218] Key type lgssc registered [ 5030.985511] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5041.864526] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5048.447220] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5048.457735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5049.551328] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5049.566550] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5049.607363] Lustre: lustre-MDT0000: new disk, initializing [ 5049.630641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5049.637628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5051.288534] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5055.878445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5056.005758] Lustre: lustre-OST0000: new disk, initializing [ 5056.007901] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5056.009982] Lustre: Skipped 1 previous similar message [ 5056.012759] Lustre: 311496:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5056.046986] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5058.241830] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5058.760622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5058.765624] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5058.780471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5279.200248] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5279.205461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5282.118589] Lustre: server umount lustre-MDT0000 complete [ 5283.266905] LustreError: 310683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129124 with bad export cookie 11837570735830872115 [ 5283.268421] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5283.270406] LustreError: 310683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5283.324803] Lustre: server umount lustre-OST0000 complete [ 5291.467492] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 5294.662071] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5306.066886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5306.168822] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5306.178983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5306.217806] Lustre: lustre-MDT0000: new disk, initializing [ 5306.240267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5306.246659] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5307.510742] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5311.489298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5311.532174] Lustre: 315725:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5311.554442] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5311.556354] Lustre: Skipped 1 previous similar message [ 5311.591579] Lustre: lustre-MDT0001: new disk, initializing [ 5311.626317] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5311.638963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5311.642843] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5312.990188] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5316.595406] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5318.270655] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5320.264685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5320.351505] Lustre: lustre-OST0000: new disk, initializing [ 5320.353484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5320.356473] Lustre: 317692:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5320.383463] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5322.107885] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5322.486258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5322.489399] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5322.517631] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5325.607183] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5327.005585] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5327.074163] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5328.400520] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5328.493491] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5331.229137] Lustre: server umount lustre-OST0000 complete [ 5332.960197] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5332.964893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5337.567947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5337.571249] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5337.576800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5342.203719] Lustre: server umount lustre-MDT0000 complete [ 5343.329950] LustreError: 315716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129184 with bad export cookie 11837570735833812892 [ 5343.332484] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5343.333439] LustreError: 315716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5353.996163] Lustre: server umount lustre-MDT0001 complete [ 5354.989732] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5355.526347] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:26:35 (1784129195) [ 5357.567161] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5357.699416] LustreError: 320005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5357.707853] LustreError: 320005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5357.719752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5358.939574] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5361.474602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5362.859356] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5365.099717] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5366.835589] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5368.898027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5370.805672] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5373.098110] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5375.215112] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5375.274797] Lustre: lustre-OST0001: new disk, initializing [ 5375.277394] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5375.281516] Lustre: 322838:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5375.309777] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5375.313515] Lustre: Skipped 2 previous similar messages [ 5377.092972] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5383.166990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5383.170169] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5383.185975] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5383.189889] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5384.226139] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5384.229221] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5385.251138] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5385.254979] LustreError: 323410:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5385.938669] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5385.999240] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5387.297873] LustreError: 323414:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5387.301979] LustreError: 323414:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5387.692109] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff983e072e4000.ost_server_uuid 50 [ 5388.144671] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff983e072e4000.ost_server_uuid in FULL state after 0 sec [ 5389.735333] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff983e072e4000.ost_server_uuid 50 [ 5390.177959] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff983e072e4000.ost_server_uuid in FULL state after 0 sec [ 5391.403786] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5403.618096] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5403.619654] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5403.624022] Lustre: Skipped 1 previous similar message [ 5403.628281] Lustre: Skipped 3 previous similar messages [ 5407.740816] Lustre: server umount lustre-OST0001 complete [ 5408.737386] LustreError: 321827:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5408.742407] LustreError: 321827:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5413.344088] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5413.344126] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5413.346324] LustreError: Skipped 1 previous similar message [ 5413.349447] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5413.352142] Lustre: Skipped 2 previous similar messages [ 5413.858112] LustreError: 321827:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5413.865374] LustreError: 321827:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5415.178829] Lustre: server umount lustre-OST0000 complete [ 5417.951840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5422.556781] Lustre: server umount lustre-MDT0000 complete [ 5423.733115] LustreError: 319987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129264 with bad export cookie 11837570735833813760 [ 5423.735128] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5423.738068] LustreError: 319987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5423.853053] Lustre: server umount lustre-MDT0001 complete [ 5427.249316] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 5428.212883] Key type lgssc unregistered [ 5428.339401] LNet: 325521:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5428.343122] LNetError: 325521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5428.352334] LNet: Removed LNI 192.168.203.139@tcp [ 5428.648116] Key type .llcrypt unregistered [ 5428.649352] Key type ._llcrypt unregistered [ 5437.150768] Key type ._llcrypt registered [ 5437.151841] Key type .llcrypt registered [ 5437.182591] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5437.497423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5437.528261] alg: No test for adler32 (adler32-zlib) [ 5438.374961] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 5438.455181] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 5440.031126] Key type lgssc registered [ 5440.354518] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5442.725544] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5442.731854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5443.860688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5445.093944] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5447.307084] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5449.375050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5449.505122] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5450.766268] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5453.649973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5453.749327] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5455.512301] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5457.720403] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5459.881697] LustreError: 328592:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5459.890920] LustreError: 328592:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5459.898528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5459.975830] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5461.939125] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5464.083668] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5468.138042] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5468.138310] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5471.115184] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5471.201882] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5546.976119] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5546.979488] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5546.985267] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5550.052065] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5550.053589] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5550.056733] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5551.608798] Lustre: server umount lustre-OST0001 complete [ 5555.172516] LustreError: 329595:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5555.172878] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5555.173244] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5555.174067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5555.174070] Lustre: Skipped 1 previous similar message [ 5555.176192] LustreError: 329595:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5559.115974] Lustre: server umount lustre-OST0000 complete [ 5562.335993] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5562.339138] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5562.344094] Lustre: Skipped 1 previous similar message [ 5562.346430] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5562.348182] Lustre: Skipped 1 previous similar message [ 5562.463428] Lustre: server umount lustre-MDT0000 complete [ 5563.587575] LustreError: 326898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129404 with bad export cookie 17794542271244666297 [ 5563.590901] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5563.591414] LustreError: 326898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5563.713302] Lustre: server umount lustre-MDT0001 complete [ 5566.895903] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 5567.803686] Key type lgssc unregistered [ 5567.927481] LNet: 331849:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5567.929592] LNetError: 331849:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5567.939240] LNet: Removed LNI 192.168.203.139@tcp [ 5568.209147] Key type .llcrypt unregistered [ 5568.210727] Key type ._llcrypt unregistered [ 5585.176232] Key type ._llcrypt registered [ 5585.177126] Key type .llcrypt registered [ 5585.206853] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5585.485529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5585.499221] alg: No test for adler32 (adler32-zlib) [ 5586.345530] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 5586.423285] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 5587.999115] Key type lgssc registered [ 5588.290175] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5591.081731] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5591.088240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5592.210588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5593.404684] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5595.982806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5596.090175] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5597.289748] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5598.157170] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5600.218398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5600.348661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5602.074419] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5602.403862] LustreError: 335602:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5602.407675] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5602.409681] LustreError: 335602:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5604.628118] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5604.703566] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5605.737545] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5606.452287] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5610.329460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5612.779309] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5616.098726] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5616.101467] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5616.104459] Lustre: Skipped 3 previous similar messages [ 5616.106025] Lustre: Skipped 2 previous similar messages [ 5621.222179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5621.225284] Lustre: Skipped 4 previous similar messages [ 5622.103524] Lustre: server umount lustre-MDT0000 complete [ 5623.158380] LustreError: 337118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129464 with bad export cookie 8346401512680890291 [ 5623.160631] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5623.162040] LustreError: 337118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5623.262743] Lustre: server umount lustre-MDT0001 complete [ 5634.566394] Lustre: server umount lustre-OST0000 complete [ 5645.979058] Lustre: server umount lustre-OST0001 complete [ 5651.017111] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5654.152661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5654.275378] LustreError: 339368:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5654.291131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5655.593658] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5658.304150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5659.814138] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5660.754064] Lustre: 340511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5662.859035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5662.979993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5662.983453] Lustre: Skipped 1 previous similar message [ 5664.734073] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5665.060499] LustreError: 340864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5665.064228] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5667.422105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5668.520908] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5669.342540] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5672.266794] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5673.324071] Lustre: 342378:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5679.075359] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5679.076487] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5679.076941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5679.080572] Lustre: Skipped 3 previous similar messages [ 5680.345790] Lustre: server umount lustre-MDT0000 complete [ 5681.400019] LustreError: 340113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129522 with bad export cookie 8346401512680891950 [ 5681.403696] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5681.403811] LustreError: 340113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5681.503137] Lustre: server umount lustre-MDT0001 complete [ 5692.423752] Lustre: server umount lustre-OST0000 complete [ 5703.901945] Lustre: server umount lustre-OST0001 complete [ 5705.057730] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5705.570643] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5706.120851] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:32:26 (1784129546) [ 5713.554292] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5716.837114] loop0: detected capacity change from 0 to 1099511627776 [ 5717.002908] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5717.005578] *** VALIDATE xfs *** [ 5717.010347] XFS (loop0): Mounting V5 Filesystem [ 5717.019527] XFS (loop0): Ending clean mount [ 5717.458034] loop1: detected capacity change from 0 to 18691697672192 [ 5737.639120] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 5740.613704] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5751.126491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5751.199471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5751.209732] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5751.242479] Lustre: lustre-MDT0000: new disk, initializing [ 5751.259243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5751.260883] Lustre: Skipped 1 previous similar message [ 5751.264779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5752.493163] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5756.345208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5756.382995] Lustre: 347489:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5756.403988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5756.406922] Lustre: Skipped 1 previous similar message [ 5756.439654] Lustre: lustre-MDT0001: new disk, initializing [ 5756.477391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5756.480048] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5757.739055] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5761.206747] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5762.912253] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5765.083260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5765.194919] Lustre: lustre-OST0000: new disk, initializing [ 5765.197172] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5765.199651] Lustre: 349455:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5767.027614] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5767.155636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5767.158390] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5767.189845] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5770.560610] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5771.910835] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5771.981909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5773.289573] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5773.361158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5776.014355] Lustre: server umount lustre-OST0000 complete [ 5777.377265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5777.377985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5777.383038] Lustre: Skipped 1 previous similar message [ 5777.387283] Lustre: Skipped 4 previous similar messages [ 5782.496614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5782.498533] Lustre: Skipped 1 previous similar message [ 5787.643949] Lustre: server umount lustre-MDT0000 complete [ 5788.805573] LustreError: 347481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129629 with bad export cookie 8346401512680893476 [ 5788.809946] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5788.810203] LustreError: 347481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5798.418887] Lustre: server umount lustre-MDT0001 complete [ 5800.860177] XFS (loop0): Unmounting Filesystem [ 5802.333240] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:34:02 (1784129642) [ 5804.423899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5804.567185] LustreError: 352009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5804.590127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5804.592080] Lustre: Skipped 2 previous similar messages [ 5805.876413] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5808.619882] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5809.988076] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5812.206126] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5813.841803] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5815.915930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5817.851955] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5820.090906] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5834.720141] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5834.722451] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5834.726428] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5834.728282] Lustre: Skipped 2 previous similar messages [ 5835.746219] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5839.694754] Lustre: server umount lustre-OST0000 complete [ 5844.447974] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5844.452116] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5844.458645] Lustre: Skipped 1 previous similar message [ 5851.104224] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5851.106808] Lustre: Skipped 6 previous similar messages [ 5855.199189] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5855.264112] Lustre: server umount lustre-MDT0000 complete [ 5856.223877] LustreError: 352009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5856.230267] LustreError: 352009:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5856.462491] LustreError: 351990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784129697 with bad export cookie 8346401512680894365 [ 5856.465601] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5856.470531] LustreError: 351990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5856.576759] Lustre: server umount lustre-MDT0001 complete [ 5859.999265] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 5860.972683] Key type lgssc unregistered [ 5861.099306] LNet: 355654:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5861.101546] LNetError: 355654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5861.107230] LNet: Removed LNI 192.168.203.139@tcp [ 5861.403085] Key type .llcrypt unregistered [ 5861.403971] Key type ._llcrypt unregistered [ 5871.369992] Key type ._llcrypt registered [ 5871.371922] Key type .llcrypt registered [ 5871.407144] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 5871.762442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5871.782733] alg: No test for adler32 (adler32-zlib) [ 5872.650522] Lustre: Lustre: Build Version: 2.17.54_163_geaacf0f [ 5872.734899] LNet: Added LNI 192.168.203.139@tcp [8/256/0/180] [ 5874.319125] Key type lgssc registered [ 5874.695485] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5877.111385] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5877.117745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5878.256332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5879.532766] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5882.237465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5882.357209] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5883.675961] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5885.966624] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5887.636289] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5889.744628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5889.859741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5891.894190] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5894.441867] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5910.021414] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.39@tcp (stopping) [ 5913.058540] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5913.062769] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5913.071902] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5913.580358] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5914.796900] Lustre: server umount lustre-MDT0001 complete [ 5915.118867] LustreError: 357192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5915.129654] LustreError: 357192:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5918.691682] LustreError: 357192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5918.700602] LustreError: 357192:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5920.237633] LustreError: 357194:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5921.181441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5921.243859] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5921.248359] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5921.275931] Lustre: 357184:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5921.482068] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5924.434517] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 5926.892354] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5927.910768] LustreError: 360823:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5927.914425] LustreError: 360823:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5927.919231] LustreError: 357189:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5927.946538] LustreError: 360826:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -17 [ 5927.951777] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5927.951777] [ 6228.960721] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6228.964806] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6228.973768] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6228.976502] Lustre: Skipped 1 previous similar message [ 6232.032285] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6232.036660] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6233.705734] Lustre: server umount lustre-MDT0001 complete [ 6237.155233] LustreError: 357197:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6237.164161] LustreError: 357197:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6239.020526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6239.070789] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6239.074437] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6239.099532] Lustre: 357187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6239.275278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6241.488553] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 6244.330992] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6249.460673] LustreError: 372592:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6249.466682] LustreError: 372592:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6249.475660] LustreError: 357189:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6249.481127] LustreError: 357189:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6249.509410] LustreError: 372603:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6249.516826] LustreError: 372603:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6249.522118] LustreError: 372603:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -17 [ 6249.527198] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.139@tcp [ 6249.527198] [ 6546.401786] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6546.402785] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6546.415278] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6546.418467] Lustre: Skipped 3 previous similar messages [ 6551.454763] Lustre: server umount lustre-MDT0001 complete [ 6551.524104] LustreError: 368064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6551.531433] LustreError: 368064:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6555.276510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6555.325831] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6555.328319] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6555.349043] Lustre: 357185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6555.481801] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6557.037504] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 6560.742274] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6565.350827] LustreError: 384625:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6565.356958] LustreError: 384625:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6565.362364] LustreError: 357189:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6565.367133] LustreError: 357189:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6565.388630] LustreError: 384628:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6565.395765] LustreError: 384628:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.139@tcp: cfg command failed: rc = -114 [ 6565.399885] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6565.399885] [ 6873.058655] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6873.059824] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6873.068629] Lustre: Skipped 1 previous similar message [ 6873.071590] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6873.074554] Lustre: Skipped 3 previous similar messages [ 6877.788765] Lustre: server umount lustre-MDT0000 complete [ 6878.176990] LustreError: 358638:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6878.182736] LustreError: 358638:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6879.078295] LustreError: 357177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784130719 with bad export cookie 4974675514841391602 [ 6879.081090] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6879.083595] LustreError: 357177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6885.233320] Lustre: server umount lustre-MDT0001 complete [ 6886.640618] Lustre: server umount lustre-OST0000 complete [ 6889.762933] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 6893.101341] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 6905.291817] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:52:25 (1784130745) [ 6907.553820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6907.660641] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6907.672266] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6907.708535] Lustre: lustre-MDT0000: new disk, initializing [ 6907.731975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6907.738989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6909.109658] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 6913.400254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6913.445543] Lustre: 398885:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6913.470973] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6913.473370] Lustre: Skipped 1 previous similar message [ 6913.511499] Lustre: lustre-MDT0001: new disk, initializing [ 6913.548900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6913.561841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6913.565432] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6915.019835] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 6918.864100] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6920.821107] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6923.218323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6923.329371] Lustre: lustre-OST0000: new disk, initializing [ 6923.331485] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6923.334360] Lustre: 400856:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6923.363913] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6924.096478] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6924.100554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6924.135655] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6925.372169] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_default_debug -1 all [ 6929.148976] Lustre: DEBUG MARKER: oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6933.983970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6933.987941] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6933.993646] Lustre: Skipped 3 previous similar messages [ 6933.996196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6933.998866] Lustre: Skipped 2 previous similar messages [ 6938.018381] Lustre: server umount lustre-MDT0000 complete [ 6939.329519] LustreError: 400855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784130780 with bad export cookie 4974675514841393541 [ 6939.331418] LustreError: 399693:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6939.333117] LustreError: MGC192.168.203.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6939.333400] LustreError: 400855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6939.452558] Lustre: server umount lustre-MDT0001 complete [ 6951.072016] Lustre: server umount lustre-OST0000 complete [ 6961.422925] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing set_hostid [ 6965.261487] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing load_modules_local [ 6976.500302] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6793 sec ========== 11:53:36 (1784130816) [ 6977.154348] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:53:37 (1784130817) === [ 6978.353182] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:53:38 (1784130818) === [ 6992.270533] Lustre: DEBUG MARKER: oleg339-server.virtnet: executing unload_modules_local [ 6993.386166] Key type lgssc unregistered [ 6993.520548] LNet: 407207:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6993.524511] LNetError: 407207:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6993.537417] LNet: Removed LNI 192.168.203.139@tcp [ 6993.860641] Key type .llcrypt unregistered [ 6993.862191] Key type ._llcrypt unregistered