[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 471954719 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003179] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008593] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.010022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.012007] pid_max: default: 32768 minimum: 301 [ 0.013123] LSM: Security Framework initializing [ 0.015009] Yama: becoming mindful. [ 0.016036] SELinux: Initializing. [ 0.017088] *** VALIDATE selinux *** [ 0.024820] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029454] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030152] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031116] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032119] *** VALIDATE tmpfs *** [ 0.034195] *** VALIDATE proc *** [ 0.035295] *** VALIDATE cgroup *** [ 0.036012] *** VALIDATE cgroup2 *** [ 0.037277] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038149] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040027] Spectre V2 : User space: Vulnerable [ 0.041008] Speculative Store Bypass: Vulnerable [ 0.045236] debug: unmapping init [mem 0xffffffff8aa59000-0xffffffff8aa60fff] [ 0.047178] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048580] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049027] ... version: 2 [ 0.050012] ... bit width: 48 [ 0.050901] ... generic registers: 4 [ 0.051011] ... value mask: 0000ffffffffffff [ 0.052012] ... max period: 00007fffffffffff [ 0.053009] ... fixed-purpose events: 3 [ 0.053822] ... event mask: 000000070000000f [ 0.054322] rcu: Hierarchical SRCU implementation. [ 0.056229] smp: Bringing up secondary CPUs ... [ 0.057578] x86: Booting SMP configuration: [ 0.058036] .... node #0, CPUs: #1 #2 #3 [ 0.061241] smp: Brought up 1 node, 4 CPUs [ 0.063015] smpboot: Max logical packages: 1 [ 0.064023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.149090] node 0 deferred pages initialised in 82ms [ 0.152098] devtmpfs: initialized [ 0.153360] x86/mm: Memory block size: 128MB [ 0.157785] gcov: version magic: 0x41383552 [ 0.158632] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.162087] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.163233] pinctrl core: initialized pinctrl subsystem [ 0.164205] [ 0.164865] ************************************************************* [ 0.165016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166014] ** ** [ 0.167016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.168013] ** ** [ 0.169015] ** This means that this kernel is built to expose internal ** [ 0.170018] ** IOMMU data structures, which may compromise security on ** [ 0.171015] ** your system. ** [ 0.172013] ** ** [ 0.173016] ** If you see this message and you are not debugging the ** [ 0.174014] ** kernel, report this immediately to your vendor! ** [ 0.175014] ** ** [ 0.176018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.177014] ************************************************************* [ 0.178731] NET: Registered protocol family 16 [ 0.179467] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.181069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.183045] cpuidle: using governor menu [ 0.185018] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.188551] PCI: Using configuration type 1 for base access [ 0.190135] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.200124] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.201022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.203087] cryptd: max_cpu_qlen set to 1000 [ 0.206288] ACPI: Added _OSI(Module Device) [ 0.207015] ACPI: Added _OSI(Processor Device) [ 0.208014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.209015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.213042] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.216124] ACPI: Interpreter enabled [ 0.217065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.218013] ACPI: Using IOAPIC for interrupt routing [ 0.219145] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.220425] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.230179] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.231047] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.232042] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.233094] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.235415] acpiphp: Slot [2] registered [ 0.236124] acpiphp: Slot [5] registered [ 0.237115] acpiphp: Slot [6] registered [ 0.238125] acpiphp: Slot [7] registered [ 0.239102] acpiphp: Slot [8] registered [ 0.240097] acpiphp: Slot [9] registered [ 0.241142] acpiphp: Slot [10] registered [ 0.242148] acpiphp: Slot [3] registered [ 0.243103] acpiphp: Slot [4] registered [ 0.244096] acpiphp: Slot [11] registered [ 0.245096] acpiphp: Slot [12] registered [ 0.246100] acpiphp: Slot [13] registered [ 0.247107] acpiphp: Slot [14] registered [ 0.248124] acpiphp: Slot [15] registered [ 0.249096] acpiphp: Slot [16] registered [ 0.250110] acpiphp: Slot [17] registered [ 0.251103] acpiphp: Slot [18] registered [ 0.252116] acpiphp: Slot [19] registered [ 0.253113] acpiphp: Slot [20] registered [ 0.254129] acpiphp: Slot [21] registered [ 0.255135] acpiphp: Slot [22] registered [ 0.256128] acpiphp: Slot [23] registered [ 0.257121] acpiphp: Slot [24] registered [ 0.258110] acpiphp: Slot [25] registered [ 0.259116] acpiphp: Slot [26] registered [ 0.260177] acpiphp: Slot [27] registered [ 0.261133] acpiphp: Slot [28] registered [ 0.262116] acpiphp: Slot [29] registered [ 0.263147] acpiphp: Slot [30] registered [ 0.264139] acpiphp: Slot [31] registered [ 0.265060] PCI host bridge to bus 0000:00 [ 0.266022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.267024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.268025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.269027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.270032] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.271034] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.272213] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.273994] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.275328] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.281015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.284065] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.285020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.286024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.287020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.288712] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.289826] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.290066] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.291737] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.294015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.303018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.305853] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.309554] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.319018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.327019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.353019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.366584] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.388021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.402019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.419018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.433000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.441022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.449017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.471019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.480964] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.486018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.493018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.505017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.513283] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.526020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.542030] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.564016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.572174] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.576012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.581016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.592019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.601283] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.603836] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.605381] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.608288] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.609146] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.612457] iommu: Default domain type: Passthrough [ 0.614645] SCSI subsystem initialized [ 0.616090] ACPI: bus type USB registered [ 0.617129] usbcore: registered new interface driver usbfs [ 0.620088] usbcore: registered new interface driver hub [ 0.622185] usbcore: registered new device driver usb [ 0.623179] pps_core: LinuxPPS API ver. 1 registered [ 0.625010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.626100] PTP clock support registered [ 0.628083] EDAC MC: Ver: 3.0.0 [ 0.630126] PCI: Using ACPI for IRQ routing [ 0.633000] NetLabel: Initializing [ 0.635012] NetLabel: domain hash size = 128 [ 0.636010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.639090] NetLabel: unlabeled traffic allowed by default [ 0.641143] vgaarb: loaded [ 0.643292] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.646016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.654599] clocksource: Switched to clocksource kvm-clock [ 0.758071] VFS: Disk quotas dquot_6.6.0 [ 0.759671] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.762154] *** VALIDATE ramfs *** [ 0.763579] *** VALIDATE hugetlbfs *** [ 0.765332] pnp: PnP ACPI init [ 0.767870] pnp: PnP ACPI: found 6 devices [ 0.783088] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.785898] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.787658] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.789424] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.791441] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.793812] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.797219] NET: Registered protocol family 2 [ 0.799282] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.802542] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.804839] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.808492] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.810622] TCP: Hash tables configured (established 65536 bind 65536) [ 0.812409] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.814230] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.815937] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.817716] NET: Registered protocol family 1 [ 0.820141] RPC: Registered named UNIX socket transport module. [ 0.822508] RPC: Registered udp transport module. [ 0.823648] RPC: Registered tcp transport module. [ 0.825428] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.827707] NET: Registered protocol family 44 [ 0.829330] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.831061] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.832576] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.834409] PCI: CLS 0 bytes, default 64 [ 0.835467] Unpacking initramfs... [ 2.271854] debug: unmapping init [mem 0xffff8a2bbcc54000-0xffff8a2bbffbffff] [ 2.275528] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.277461] software IO TLB: mapped [mem 0x00000000a3400000-0x00000000a7400000] (64MB) [ 2.279859] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.775421] Initialise system trusted keyrings [ 2.777058] Key type blacklist registered [ 2.779189] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.788459] zbud: loaded [ 2.791310] *** VALIDATE nfs *** [ 2.792637] *** VALIDATE nfs4 *** [ 2.794462] pstore: using deflate compression [ 2.798443] Platform Keyring initialized [ 2.905831] NET: Registered protocol family 38 [ 2.908182] Key type asymmetric registered [ 2.909944] Asymmetric key parser 'x509' registered [ 2.912333] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.915809] io scheduler mq-deadline registered [ 2.917714] io scheduler kyber registered [ 2.919497] io scheduler bfq registered [ 2.921528] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.924832] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.928138] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.931054] ACPI: Power Button [PWRF] [ 2.935824] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.941570] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.952899] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.959632] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.973378] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.001672] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.028497] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.032682] Non-volatile memory driver v1.3 [ 3.033966] Linux agpgart interface v0.103 [ 3.064379] virtio_blk virtio1: [vda] 136464 512-byte logical blocks (69.9 MB/66.6 MiB) [ 3.066611] vda: detected capacity change from 0 to 69869568 [ 3.085425] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.088332] vdb: detected capacity change from 0 to 1073741824 [ 3.105105] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.108836] vdc: detected capacity change from 0 to 2621440000 [ 3.132830] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.136492] vdd: detected capacity change from 0 to 2621440000 [ 3.153915] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.157239] vde: detected capacity change from 0 to 4294967296 [ 3.173409] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.176765] vdf: detected capacity change from 0 to 4294967296 [ 3.187163] libphy: Fixed MDIO Bus: probed [ 3.199514] usbcore: registered new interface driver usbserial_generic [ 3.202180] usbserial: USB Serial support registered for generic [ 3.204984] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.210097] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.212418] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.214977] mousedev: PS/2 mouse device common for all mice [ 3.217961] rtc_cmos 00:05: RTC can wake from S4 [ 3.221721] rtc_cmos 00:05: registered as rtc0 [ 3.224210] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.224374] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.227362] intel_pstate: CPU model not supported [ 3.234340] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.237375] hid: raw HID events driver (C) Jiri Kosina [ 3.240220] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.240982] usbcore: registered new interface driver usbhid [ 3.246761] usbhid: USB HID core driver [ 3.248379] drop_monitor: Initializing network drop monitor service [ 3.250955] Initializing XFRM netlink socket [ 3.253165] NET: Registered protocol family 10 [ 3.256812] Segment Routing with IPv6 [ 3.258439] NET: Registered protocol family 17 [ 3.260913] mpls_gso: MPLS GSO support [ 3.267634] RAS: Correctable Errors collector initialized. [ 3.270122] AVX version of gcm_enc/dec engaged. [ 3.271952] AES CTR mode by8 optimization enabled [ 3.358560] sched_clock: Marking stable (3358482584, 0)->(4335312037, -976829453) [ 3.362890] registered taskstats version 1 [ 3.364861] Loading compiled-in X.509 certificates [ 3.367628] zswap: loaded using pool lzo/zbud [ 3.394269] Key type big_key registered [ 3.406726] Key type encrypted registered [ 3.408598] ima: No TPM chip found, activating TPM-bypass! [ 3.410559] ima: Allocated hash algorithm: sha1 [ 3.412629] ima: No architecture policies found [ 3.415711] evm: Initialising EVM extended attributes: [ 3.417899] evm: security.selinux [ 3.419557] evm: security.ima [ 3.420763] evm: security.capability [ 3.422217] evm: HMAC attrs: 0x1 [ 3.424843] rtc_cmos 00:05: setting system clock to 2026-06-12 02:01:35 UTC (1781229695) [ 3.431521] debug: unmapping init [mem 0xffffffff8ba03000-0xffffffff8bbfffff] [ 3.434887] debug: unmapping init [mem 0xffffffff8a782000-0xffffffff8aa58fff] [ 3.444126] Write protecting the kernel read-only data: 28672k [ 3.447833] debug: unmapping init [mem 0xffffffff88e03000-0xffffffff88ffffff] [ 3.450851] debug: unmapping init [mem 0xffffffff89714000-0xffffffff897fffff] [ 3.485642] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.495028] systemd[1]: Detected virtualization kvm. [ 3.496728] systemd[1]: Detected architecture x86-64. [ 3.498183] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.523470] systemd[1]: No hostname configured. [ 3.524600] systemd[1]: Set hostname to . [ 3.525976] random: systemd: uninitialized urandom read (16 bytes read) [ 3.527702] systemd[1]: Initializing machine ID from random generator. [ 3.666539] random: systemd: uninitialized urandom read (16 bytes read) [ 3.669485] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.674129] random: systemd: uninitialized urandom read (16 bytes read) [ 3.677568] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.687349] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.349339] device-mapper: uevent: version 1.0.3 [ 4.351853] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.140818] random: fast init done [ 5.165181] virtio_net virtio0 ens2: renamed from eth0 [ 5.245829] scsi host0: ata_piix [ 5.263924] scsi host1: ata_piix [ 5.285550] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.288055] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.557966] dracut-initqueue[580]: RTNETLINK answers: File exists [ 9.935382] random: crng init done [ 9.937632] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 12.436348] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.245302] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.000282] SELinux: Disabled at runtime. [ 16.172935] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.190626] systemd[1]: Detected virtualization kvm. [ 16.193469] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 18.379600] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.387860] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.403207] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.411817] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.429457] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.441563] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.459581] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Listening on udev Control Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ 19.116672] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Mounted Huge Pages File System. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 20.184283] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.256431] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.457933] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.943941] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.026624] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.826460] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 28.762898] NFS: Registering the id_resolver key type [ 28.765735] Key type id_resolver registered [ 28.768089] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg212-server login: [ 87.013609] libcfs: loading out-of-tree module taints kernel. [ 87.058665] Key type ._llcrypt registered [ 87.065807] Key type .llcrypt registered [ 87.209636] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 107.285620] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 108.975962] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 109.032807] alg: No test for adler32 (adler32-zlib) [ 110.490980] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 111.568263] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 113.359169] Key type lgssc registered [ 115.935622] Lustre: Echo OBD driver; http://www.lustre.org/ [ 139.608836] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 186.474782] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 190.713128] hrtimer: interrupt took 5164382 ns [ 204.244649] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 204.304758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 205.800549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 205.856977] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 206.035265] Lustre: lustre-MDT0000: new disk, initializing [ 206.223450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 206.254511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 212.133764] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 229.134271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 229.245833] Lustre: 6507:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 229.300660] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 229.312208] Lustre: Skipped 1 previous similar message [ 229.466617] Lustre: lustre-MDT0001: new disk, initializing [ 229.623741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 229.675469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 229.695890] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 236.132166] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 242.542678] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 254.269512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 254.728597] Lustre: lustre-OST0000: new disk, initializing [ 254.742975] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 254.754745] Lustre: 8412:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 254.898547] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 262.756505] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 262.764092] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 262.866868] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 263.454556] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 279.500529] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 279.736321] Lustre: lustre-OST0001: new disk, initializing [ 279.739808] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 279.748330] Lustre: 9470:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 279.839481] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 285.289187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 285.299867] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 285.420795] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 287.545158] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 300.333112] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 307.773774] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 314.945965] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing check_logdir /tmp/testlogs/ [ 320.866964] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing yml_node [ 325.424492] Lustre: DEBUG MARKER: Client: 2.17.53.61 [ 328.185899] Lustre: DEBUG MARKER: MDS: 2.17.53.61 [ 330.806612] Lustre: DEBUG MARKER: OSS: 2.17.53.61 [ 332.397329] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jun 11 22:07:03 EDT 2026 [ 351.494798] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 353.432809] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 372.705069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.710833] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 372.732500] Lustre: Skipped 3 previous similar messages [ 372.741852] Lustre: Skipped 2 previous similar messages [ 375.741317] Lustre: server umount lustre-MDT0000 complete [ 379.573464] LustreError: 8422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230071 with bad export cookie 16688360904088081432 [ 379.578069] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 379.584496] LustreError: 8422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 382.945627] LustreError: 9413:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 382.946310] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 382.969513] LustreError: 9413:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 382.997653] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 383.027190] Lustre: Skipped 2 previous similar messages [ 384.995077] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 384.997033] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 385.033060] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 385.056522] Lustre: Skipped 1 previous similar message [ 386.017446] Lustre: server umount lustre-MDT0001 complete [ 391.103783] Lustre: server umount lustre-OST0000 complete [ 395.811708] Lustre: server umount lustre-OST0001 complete [ 405.176679] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 418.964218] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 459.949474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 460.314194] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 460.343635] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 460.473332] Lustre: lustre-MDT0000: new disk, initializing [ 460.582694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 460.607249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 465.507985] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 476.453037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 476.535848] Lustre: 14987:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 476.586563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 476.590394] Lustre: Skipped 1 previous similar message [ 476.651933] Lustre: lustre-MDT0001: new disk, initializing [ 476.737659] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 476.774924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 476.789466] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 481.154198] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 489.231892] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 493.541026] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 501.546535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 501.870676] Lustre: lustre-OST0000: new disk, initializing [ 501.878952] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 501.894639] Lustre: 16729:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 502.012707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 503.917631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 503.928845] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 504.040679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 509.623366] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 518.944875] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 525.272912] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 525.586649] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 530.687028] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 531.023994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 535.009143] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 535.009504] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.024590] LustreError: Skipped 1 previous similar message [ 535.025868] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 535.045063] Lustre: Skipped 2 previous similar messages [ 539.856437] Lustre: server umount lustre-OST0000 complete [ 548.322734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 548.332140] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 548.339275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 548.342354] Lustre: Skipped 1 previous similar message [ 552.128106] Lustre: server umount lustre-MDT0000 complete [ 555.491067] LustreError: 17057:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 555.511070] LustreError: 17057:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 557.017392] LustreError: 16728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230249 with bad export cookie 16688360904088082965 [ 557.020845] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 557.027420] LustreError: 16728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 557.360938] Lustre: server umount lustre-MDT0001 complete [ 563.936853] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 22:10:54 (1781230254) [ 571.922895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 572.510724] LustreError: 18848:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.605865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 577.902791] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 578.032100] LustreError: 18849:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 587.252178] LustreError: 18849:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 587.266068] LustreError: 18849:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 588.124779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 594.069922] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 599.842531] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 605.014839] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 612.559513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 619.701437] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 625.843949] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 629.513919] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 637.921801] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 637.925037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 637.947647] Lustre: Skipped 3 previous similar messages [ 637.966260] Lustre: Skipped 4 previous similar messages [ 643.253210] Lustre: server umount lustre-MDT0000 complete [ 646.883753] LustreError: 18829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230338 with bad export cookie 16688360904088083847 [ 646.884733] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 646.893743] LustreError: 18829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 647.170645] Lustre: server umount lustre-MDT0001 complete [ 660.154266] Lustre: server umount lustre-OST0000 complete [ 682.106248] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 693.586460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 693.933958] LustreError: 22874:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 693.992901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 693.998182] Lustre: Skipped 2 previous similar messages [ 698.457351] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 707.949021] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 712.298294] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 715.019128] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 721.956237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 728.234357] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 736.921565] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 737.093727] Lustre: lustre-OST0001: new disk, initializing [ 737.098509] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 737.104872] Lustre: 25138:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 739.054569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 739.065969] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 739.158469] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 743.483366] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 754.571079] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 758.749833] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 771.043581] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 771.066515] Lustre: Skipped 1 previous similar message [ 771.074811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 771.109511] Lustre: Skipped 6 previous similar messages [ 776.908834] Lustre: server umount lustre-MDT0000 complete [ 780.665126] LustreError: 23980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230472 with bad export cookie 16688360904088084694 [ 780.688182] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 780.689717] LustreError: 23980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 781.119657] Lustre: server umount lustre-MDT0001 complete [ 795.066775] Lustre: server umount lustre-OST0000 complete [ 797.153168] Lustre: 3653:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781230473/real 1781230473] req@ffff8a2b04a2ce00 x1867754822158720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781230489 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 797.182402] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 798.575417] Lustre: server umount lustre-OST0001 complete [ 816.709180] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 828.482564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 828.817034] LustreError: 28123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 828.832742] LustreError: 28123:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 828.924908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 828.930454] Lustre: Skipped 3 previous similar messages [ 833.643700] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 844.982983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 851.598282] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 855.091089] Lustre: 29236:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 861.962685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 868.048734] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 877.512470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 885.556931] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 893.539292] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 897.769976] Lustre: 31084:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 907.232612] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 907.242386] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 907.253012] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 907.258444] Lustre: Skipped 4 previous similar messages [ 910.472024] Lustre: server umount lustre-MDT0000 complete [ 913.897777] LustreError: 28124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 913.907022] LustreError: 28124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 914.216222] LustreError: 28103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230606 with bad export cookie 16688360904088085989 [ 914.218353] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 914.233706] LustreError: 28103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 914.672895] Lustre: server umount lustre-MDT0001 complete [ 928.341974] Lustre: server umount lustre-OST0000 complete [ 942.420264] Lustre: server umount lustre-OST0001 complete [ 951.449806] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 22:17:22 (1781230642) [ 979.962897] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 989.879258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 994.931706] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1004.023876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1009.074269] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1011.976117] Lustre: 34989:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1018.607622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1025.092667] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1033.833989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1040.066325] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1047.860877] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1051.900583] Lustre: 36833:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1060.838740] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1060.869633] Lustre: Skipped 3 previous similar messages [ 1060.878338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1060.894551] Lustre: Skipped 7 previous similar messages [ 1064.111743] Lustre: server umount lustre-MDT0000 complete [ 1065.952413] LustreError: 34593:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1065.992683] LustreError: 34593:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 1068.474577] LustreError: 33856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230760 with bad export cookie 16688360904088087095 [ 1068.479420] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1068.493743] LustreError: 33856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1068.878926] Lustre: server umount lustre-MDT0001 complete [ 1083.859931] Lustre: server umount lustre-OST0000 complete [ 1087.312718] Lustre: 3654:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781230763/real 1781230763] req@ffff8a2c08d42300 x1867754822258560/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781230779 ref 1 fl Rpc:EXNQr/200/ffffffff rc -5/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1097.793324] Lustre: server umount lustre-OST0001 complete [ 1107.753603] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 22:19:58 (1781230798) [ 1125.422393] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1136.351739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1136.674379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1136.682457] Lustre: Skipped 7 previous similar messages [ 1140.618993] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1149.843638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1155.334732] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1158.444475] Lustre: 40072:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1165.959886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1172.855851] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1182.673430] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1189.104713] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1195.788199] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1199.206396] Lustre: 41915:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1200.089631] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1200.094498] Lustre: Skipped 1 previous similar message [ 1203.683916] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1203.687934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1203.707687] Lustre: Skipped 5 previous similar messages [ 1203.717781] Lustre: Skipped 3 previous similar messages [ 1209.477819] Lustre: server umount lustre-MDT0000 complete [ 1212.960232] LustreError: 38939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230905 with bad export cookie 16688360904088088201 [ 1212.966619] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1212.971318] LustreError: 38939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1219.434929] Lustre: server umount lustre-MDT0001 complete [ 1227.174074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1242.884919] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1249.268067] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1252.457054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1257.177339] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1258.493087] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1258.510735] Lustre: Skipped 1 previous similar message [ 1264.657340] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1266.907738] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1271.613602] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1273.366572] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1284.067150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1285.419468] Lustre: server umount lustre-MDT0000 complete [ 1289.210952] LustreError: 41245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230981 with bad export cookie 16688360904088089328 [ 1289.213935] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1289.217076] LustreError: 41245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1289.597495] Lustre: server umount lustre-MDT0001 complete [ 1304.751337] Lustre: server umount lustre-OST0000 complete [ 1318.853562] Lustre: server umount lustre-OST0001 complete [ 1329.156918] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 22:23:40 (1781231020) [ 1346.950598] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1356.599284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1356.889976] LustreError: 46285:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1356.912938] LustreError: 46285:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 28 previous similar messages [ 1361.460541] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1369.920809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.461472] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1377.478441] Lustre: 47405:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1383.891090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1393.165286] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1402.723190] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.132495] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1417.947504] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1423.134090] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1435.795374] Lustre: server umount lustre-MDT0000 complete [ 1436.640449] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1440.011097] LustreError: 46265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231132 with bad export cookie 16688360904088090119 [ 1440.018141] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1440.029704] LustreError: 46265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1442.730577] Lustre: server umount lustre-MDT0001 complete [ 1457.806917] Lustre: server umount lustre-OST0000 complete [ 1460.703292] Lustre: 3653:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781231136/real 1781231136] req@ffff8a2b04b5a300 x1867754822402432/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781231152 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1460.720313] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1460.731884] Lustre: Skipped 11 previous similar messages [ 1461.690977] Lustre: server umount lustre-OST0001 complete [ 1471.555325] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 22:26:02 (1781231162) [ 1474.618747] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1476.502930] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 22:26:07 (1781231167) [ 1488.070343] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 1497.244994] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1524.637853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1524.974314] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1525.041713] Lustre: lustre-MDT0000: new disk, initializing [ 1525.129087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1529.190656] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1538.537754] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1545.692814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1545.996369] Lustre: lustre-OST0000: new disk, initializing [ 1546.000862] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1546.005295] Lustre: Skipped 1 previous similar message [ 1546.014389] Lustre: 53298:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1548.029762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1548.035843] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1548.079115] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1554.842569] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1565.063717] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1567.789733] Lustre: DEBUG MARKER: create test files [ 1599.459209] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1599.467546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1599.475200] Lustre: Skipped 14 previous similar messages [ 1604.975836] Lustre: server umount lustre-OST0000 complete [ 1608.877805] Lustre: server umount lustre-MDT0000 complete [ 1618.682319] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1621.273875] Key type lgssc unregistered [ 1621.626508] LNet: 54801:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1621.639427] LNetError: 54801:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1621.665073] LNet: Removed LNI 192.168.202.112@tcp [ 1622.533144] Key type .llcrypt unregistered [ 1622.535073] Key type ._llcrypt unregistered [ 1652.087865] Key type ._llcrypt registered [ 1652.092793] Key type .llcrypt registered [ 1652.231052] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1653.974824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1654.006077] alg: No test for adler32 (adler32-zlib) [ 1655.170842] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 1655.451304] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1657.111641] Key type lgssc registered [ 1658.125614] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1665.941952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1665.960242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1667.509353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1672.261938] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1678.813593] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1686.157644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1686.438991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1692.603457] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1692.746416] Lustre: 57407:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1692.764268] Lustre: 57407:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1692.772933] Lustre: 57407:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1692.783806] Lustre: 57407:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1692.789570] Lustre: 57407:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1692.808364] Lustre: 57407:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1693.461756] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1699.798490] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1705.629557] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1711.543041] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1718.241459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1718.265846] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1719.976505] Lustre: server umount lustre-OST0000 complete [ 1724.529326] Lustre: server umount lustre-MDT0000 complete [ 1735.163499] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1737.745173] Key type lgssc unregistered [ 1738.010450] LNet: 58885:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1738.014426] LNetError: 58885:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1738.031579] LNet: Removed LNI 192.168.202.112@tcp [ 1738.903650] Key type .llcrypt unregistered [ 1738.906919] Key type ._llcrypt unregistered [ 1772.999939] Key type ._llcrypt registered [ 1773.001325] Key type .llcrypt registered [ 1773.152840] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1774.886694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1774.941280] alg: No test for adler32 (adler32-zlib) [ 1776.129912] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 1776.404097] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1778.071155] Key type lgssc registered [ 1779.695309] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1789.996353] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1790.043145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1791.747179] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1797.020481] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1804.256805] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1812.490513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1813.019872] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1816.136037] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1816.406115] Lustre: 61594:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1816.420366] Lustre: 61594:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1816.424261] Lustre: 61594:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1816.428838] Lustre: 61594:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1816.434052] Lustre: 61594:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1816.438922] Lustre: 61594:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1820.479295] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1827.413341] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1832.411976] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1846.752460] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1846.776357] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1848.775634] Lustre: server umount lustre-OST0000 complete [ 1853.232928] Lustre: server umount lustre-MDT0000 complete [ 1865.577111] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 1868.573376] Key type lgssc unregistered [ 1868.931381] LNet: 63021:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1868.942230] LNetError: 63021:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1868.965349] LNet: Removed LNI 192.168.202.112@tcp [ 1870.012827] Key type .llcrypt unregistered [ 1870.015301] Key type ._llcrypt unregistered [ 1889.882381] Key type ._llcrypt registered [ 1889.884903] Key type .llcrypt registered [ 1890.000870] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 1907.975702] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 1909.256640] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1909.297297] alg: No test for adler32 (adler32-zlib) [ 1910.467520] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 1910.754613] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 1912.458089] Key type lgssc registered [ 1913.696930] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1953.943429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1953.979252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1955.396958] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1955.464792] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1955.591512] Lustre: lustre-MDT0000: new disk, initializing [ 1955.724351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1955.842344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1961.267829] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1974.254551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1974.437756] Lustre: 66144:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1974.537843] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1974.546472] Lustre: Skipped 1 previous similar message [ 1974.710821] Lustre: lustre-MDT0001: new disk, initializing [ 1974.830358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1974.876764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1974.887944] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1979.975950] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 1989.228237] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1993.124856] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2002.298882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2002.609213] Lustre: lustre-OST0000: new disk, initializing [ 2002.614507] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2002.620098] Lustre: 67890:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2002.702259] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2009.731987] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2009.759253] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2009.899272] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2011.213456] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2020.574359] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2025.187897] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2025.404908] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2030.244368] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2030.453803] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2035.683523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2035.684735] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.690477] LustreError: Skipped 1 previous similar message [ 2035.719036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2035.722236] Lustre: Skipped 1 previous similar message [ 2035.743423] Lustre: Skipped 1 previous similar message [ 2038.594715] Lustre: server umount lustre-OST0000 complete [ 2045.921350] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2045.964548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2045.974727] Lustre: Skipped 1 previous similar message [ 2049.123677] Lustre: server umount lustre-MDT0000 complete [ 2053.438059] LustreError: 66135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231745 with bad export cookie 11940637450432222037 [ 2053.453803] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2053.465027] LustreError: 66135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2053.947444] Lustre: server umount lustre-MDT0001 complete [ 2064.394837] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:35:55 (1781231755) [ 2076.474326] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2125.448556] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 2136.870119] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2179.811113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2180.304369] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2180.350675] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2180.479484] Lustre: lustre-MDT0000: new disk, initializing [ 2180.622942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2180.674767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2185.049292] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2196.163226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2196.260211] Lustre: 73718:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2196.347633] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2196.353757] Lustre: Skipped 1 previous similar message [ 2196.542434] Lustre: lustre-MDT0001: new disk, initializing [ 2196.703135] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2196.736422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2196.764180] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2202.381700] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2211.851725] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2216.244387] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2224.897717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2225.227193] Lustre: lustre-OST0000: new disk, initializing [ 2225.240642] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2225.250416] Lustre: 75459:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2225.348587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2230.937050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2230.951863] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2231.071155] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2233.138478] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2243.172507] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2248.190844] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2248.396173] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2253.421288] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2253.636744] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2256.864903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2256.872844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2256.897624] Lustre: Skipped 2 previous similar messages [ 2256.909518] Lustre: Skipped 1 previous similar message [ 2262.020197] Lustre: server umount lustre-OST0000 complete [ 2267.105515] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2267.108770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2267.120612] Lustre: Skipped 1 previous similar message [ 2267.137327] Lustre: Skipped 1 previous similar message [ 2272.224553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2272.229903] Lustre: Skipped 1 previous similar message [ 2272.829358] Lustre: server umount lustre-MDT0000 complete [ 2277.352930] LustreError: 73730:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2277.386443] LustreError: 73730:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2277.494110] LustreError: 73710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231969 with bad export cookie 11940637450432222919 [ 2277.510954] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2277.513677] LustreError: 73710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2277.819909] Lustre: server umount lustre-MDT0001 complete [ 2289.074648] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:39:39 (1781231979) [ 2297.310176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.735505] LustreError: 77726:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2297.769353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2302.712752] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2302.957919] LustreError: 77727:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2307.058534] LustreError: 77726:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2312.162916] LustreError: 77727:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2313.059429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2318.237283] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2324.775734] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2328.562726] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2335.588279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2335.976236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2335.985976] Lustre: Skipped 1 previous similar message [ 2342.723842] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2347.936681] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2351.259592] Lustre: 80059:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 82c59bc2-0f1f-4cd9-9a7f-2dc0d23472ea at adminstrative request [ 2353.635347] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2353.662930] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2363.875723] Lustre: MGS: Client 82c59bc2-0f1f-4cd9-9a7f-2dc0d23472ea (at 0@lo) reconnecting [ 2363.922037] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa5b5aa7f38bd1a39 to 0xa5b5aa7f38bd1cd9 [ 2363.943091] Lustre: MGC192.168.202.112@tcp: Connection restored to 0@lo (at 0@lo) [ 2391.799250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2392.009952] Lustre: lustre-OST0001: new disk, initializing [ 2392.014711] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2392.020607] Lustre: 80454:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2399.971857] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2401.407100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2401.417635] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2401.540746] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2408.455272] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2417.126271] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2417.128387] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2417.141533] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2417.151375] Lustre: Skipped 1 previous similar message [ 2418.398865] Lustre: server umount lustre-OST0001 complete [ 2422.259593] LustreError: 79320:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2422.289161] LustreError: 79320:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2428.188857] Lustre: server umount lustre-OST0000 complete [ 2432.480187] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2432.498938] Lustre: Skipped 3 previous similar messages [ 2437.605051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2437.622403] Lustre: Skipped 8 previous similar messages [ 2438.464551] Lustre: server umount lustre-MDT0000 complete [ 2441.636590] LustreError: 77706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232133 with bad export cookie 11940637450432224473 [ 2441.651486] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2441.652802] LustreError: 77706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2441.919431] Lustre: server umount lustre-MDT0001 complete [ 2451.192262] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:42:22 (1781232142) [ 2452.864745] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2454.469240] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:42:25 (1781232145) [ 2455.959500] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2457.535814] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:42:28 (1781232148) [ 2458.729712] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2460.382874] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:42:31 (1781232151) [ 2462.119436] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2463.973608] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:42:35 (1781232155) [ 2465.699849] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2471.451979] loop: module loaded [ 2471.478905] loop0: detected capacity change from 0 to 4294967296 [ 2471.523083] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2473.204469] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2476.384953] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2480.768619] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2508.551659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2508.648702] LustreError: 84446:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2508.669660] LustreError: 84446:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2508.680822] LustreError: 84446:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2508.688571] LustreError: 84446:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2531.516285] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 2545.207712] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 2584.741026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2585.079831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2585.148119] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2585.283566] Lustre: lustre-MDT0000: new disk, initializing [ 2585.400149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2585.409752] Lustre: Skipped 1 previous similar message [ 2585.472508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2589.152210] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2600.371190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2600.506356] Lustre: 87278:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2600.857197] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2604.911433] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2611.213175] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2615.163746] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2622.365222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2622.604708] Lustre: 89022:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2628.133922] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2629.697823] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2629.800032] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2635.044431] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2639.268858] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2639.454512] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2643.834417] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2644.020692] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2650.592107] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2650.597604] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2650.604569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2650.609323] Lustre: Skipped 1 previous similar message [ 2660.319165] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2660.551893] Lustre: server umount lustre-OST0000 complete [ 2678.239682] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2678.557894] Lustre: server umount lustre-MDT0000 complete [ 2680.291581] LustreError: 87290:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2680.325270] LustreError: 87290:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2682.800969] LustreError: 87272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232374 with bad export cookie 11940637450432225089 [ 2682.815250] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2682.817335] LustreError: 87272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2683.366867] Lustre: server umount lustre-MDT0001 complete [ 2692.424785] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:46:23 (1781232383) [ 2699.940833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2705.183186] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2714.982021] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2715.106022] LustreError: 91291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2715.135200] LustreError: 91291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2715.437062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2715.439781] Lustre: Skipped 3 previous similar messages [ 2720.508527] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2727.610695] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2732.851657] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2741.408226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2748.587418] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2755.642856] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2762.234535] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2762.515796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2767.774059] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2768.027137] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2776.179105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2776.456984] Lustre: lustre-OST0001: new disk, initializing [ 2776.459893] Lustre: Skipped 2 previous similar messages [ 2776.462664] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2776.468426] Lustre: Skipped 3 previous similar messages [ 2776.486897] Lustre: 94286:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2782.766558] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2784.941332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2784.957637] Lustre: Skipped 2 previous similar messages [ 2784.970136] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2785.189544] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2790.786663] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2796.203708] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2796.467237] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2801.884402] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2802.143226] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2822.100353] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2845.454675] Lustre: Failing over lustre-MDT0000 [ 2846.045567] Lustre: server umount lustre-MDT0000 complete [ 2846.586725] LustreError: 94735:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.12@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2846.605500] LustreError: 94735:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2846.717802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2846.735364] Lustre: Skipped 5 previous similar messages [ 2862.051907] Lustre: 64367:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781232538/real 1781232538] req@0000000077c53d9b x1867756708569472/t0(0) o400->MGC192.168.202.112@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1781232554 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2862.078023] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2870.860970] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2870.863411] LDISKFS-fs (dm-0): recovery complete [ 2870.870450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2872.032778] LustreError: 96955:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2872.045648] LustreError: 96955:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000ee256a31 x1867756708575360/t0(0) o250->MGC192.168.202.112@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1781232564 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2872.072614] LustreError: 96955:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2872.288823] LustreError: 64365:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000505ad54f x1867756708576896/t0(0) o250->MGC192.168.202.112@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2872.639678] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2873.153673] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2873.155787] Lustre: 94735:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2876.710224] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 2877.925872] Lustre: 91291:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2877.935110] Lustre: 91291:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 2877.944166] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2878.028938] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.360300] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2878.681818] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.699348] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2879.015184] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2879.025075] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2879.037262] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2879.050966] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 2879.697356] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2879.720198] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2880.056120] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2880.065974] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2881.040520] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2881.058406] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 5 previous similar messages [ 2882.009738] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.023868] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2882.343169] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.357415] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2885.201630] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2885.216954] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2886.167978] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2886.178960] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2886.503136] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2886.510681] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2893.475355] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2893.502883] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 25 previous similar messages [ 2894.447994] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2894.458611] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2894.775196] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.786022] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2909.679969] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2909.697848] Lustre: 96987:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 50 previous similar messages [ 2910.666819] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2910.676774] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2910.999790] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2911.009897] LustreError: 96987:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2933.215155] Lustre: 96987:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2933.234048] Lustre: 96987:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2933.241186] Lustre: 96987:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 7375a12a-7163-424e-a5de-151ea7cb12a8@192.168.202.12@tcp [ 2933.261291] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2933.270077] Lustre: 96987:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@0000000001eabef1 x1867756691589760/t0(8589934770) o35->7375a12a-7163-424e-a5de-151ea7cb12a8@192.168.202.12@tcp:395/0 lens 392/0 e 0 to 0 dl 1781232635 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2933.290824] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2933.301980] Lustre: lustre-MDT0000: Denying connection for new client 7375a12a-7163-424e-a5de-151ea7cb12a8 (at 192.168.202.12@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2933.323150] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2933.353185] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2933.357804] Lustre: Skipped 2 previous similar messages [ 2933.358738] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2933.389589] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2933.395304] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2954.208911] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2954.223046] LustreError: Skipped 1 previous similar message [ 2954.243713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2954.272640] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2954.280508] Lustre: Skipped 9 previous similar messages [ 2955.855498] Lustre: server umount lustre-OST0000 complete [ 2964.452707] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2964.469142] LustreError: Skipped 2 previous similar messages [ 2966.575467] Lustre: server umount lustre-OST0001 complete [ 2976.278432] Lustre: server umount lustre-MDT0000 complete [ 2980.249367] LustreError: 97565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232672 with bad export cookie 11940637450432276301 [ 2980.251700] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2980.257803] LustreError: 97565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2980.701339] Lustre: server umount lustre-MDT0001 complete [ 2990.746275] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:51:21 (1781232681) [ 2998.988410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2998.996913] Lustre: *** cfs_fail_loc=197, val=0*** [ 3014.687697] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3019.807692] LustreError: 98947:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.112@tcp: failed processing log, type 4: rc = -110 [ 3024.943503] Lustre: *** cfs_fail_loc=197, val=0*** [ 3024.946398] Lustre: Skipped 65 previous similar messages [ 3045.537019] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3045.544920] Lustre: Skipped 3 previous similar messages [ 3053.289777] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3059.382044] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3061.675204] Lustre: server umount lustre-OST0000 complete [ 3070.994606] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:52:42 (1781232762) [ 3092.705895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3093.184569] LustreError: 100456:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3093.211465] LustreError: 100456:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 38 previous similar messages [ 3097.199876] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3105.265333] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3110.118748] LustreError: 100487:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3110.139826] LustreError: 100487:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3110.144490] LustreError: 100487:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3110.608751] Lustre: server umount lustre-MDT0000 complete [ 3144.092953] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 3155.209976] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3190.579981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3190.769622] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3190.792270] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3190.865290] Lustre: lustre-MDT0000: new disk, initializing [ 3190.948945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3194.814177] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3204.682659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3204.835510] Lustre: 104514:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3205.201917] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3209.109839] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3215.875718] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3219.492194] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3225.669661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3226.021825] Lustre: 106259:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3232.039621] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3235.024280] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3239.303910] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3244.027796] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3244.295582] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3248.990862] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3249.235297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3255.778771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3255.790254] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3255.792866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3255.792876] Lustre: Skipped 7 previous similar messages [ 3255.802000] Lustre: Skipped 6 previous similar messages [ 3257.784620] Lustre: server umount lustre-OST0000 complete [ 3269.149914] Lustre: server umount lustre-MDT0000 complete [ 3273.544996] LustreError: 104506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232965 with bad export cookie 11940637450432286899 [ 3273.556523] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3273.559593] LustreError: 104506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3274.027581] Lustre: server umount lustre-MDT0001 complete [ 3283.183741] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:56:14 (1781232974) [ 3295.496434] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3298.727379] Key type lgssc unregistered [ 3298.955657] LNet: 108579:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3298.968418] LNetError: 108579:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3298.987402] LNet: Removed LNI 192.168.202.112@tcp [ 3299.887967] Key type .llcrypt unregistered [ 3299.890374] Key type ._llcrypt unregistered [ 3317.718062] Key type ._llcrypt registered [ 3317.721232] Key type .llcrypt registered [ 3317.854205] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 3335.073213] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3336.383816] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3336.404053] alg: No test for adler32 (adler32-zlib) [ 3337.406962] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 3337.567761] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 3339.191191] Key type lgssc registered [ 3340.376984] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3381.533975] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3381.565146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3382.815322] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3382.843425] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3382.914236] Lustre: lustre-MDT0000: new disk, initializing [ 3383.020148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3383.041696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3387.076162] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3396.933921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3397.177265] Lustre: lustre-OST0000: new disk, initializing [ 3397.181142] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3397.183535] Lustre: Skipped 1 previous similar message [ 3397.189405] Lustre: 112702:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3397.252507] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3400.087447] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3400.099673] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3400.134349] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3403.097221] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3412.483370] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3412.582508] Lustre: lustre-OST0001: new disk, initializing [ 3412.585348] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3412.588669] Lustre: 113550:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3412.655337] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3418.310458] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3419.818620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3419.829364] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3419.874787] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3429.869282] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3429.879824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3429.899780] Lustre: Skipped 1 previous similar message [ 3429.916519] Lustre: Skipped 1 previous similar message [ 3433.731117] Lustre: server umount lustre-MDT0000 complete [ 3440.460411] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3445.860382] LustreError: 112701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233137 with bad export cookie 761355991825224158 [ 3445.866380] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3445.882247] LustreError: 112701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3446.039451] Lustre: server umount lustre-OST0000 complete [ 3449.801915] Lustre: server umount lustre-OST0001 complete [ 3458.564893] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 22:59:09 (1781233149) [ 3468.984597] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3490.160742] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:59:41 (1781233181) [ 3529.206525] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 3540.129116] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3575.049568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3575.433695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3575.482704] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3575.586895] Lustre: lustre-MDT0000: new disk, initializing [ 3575.741364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3575.780694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3580.174862] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3591.651995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3591.821460] Lustre: 120240:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3591.917457] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3591.921995] Lustre: Skipped 1 previous similar message [ 3592.051485] Lustre: lustre-MDT0001: new disk, initializing [ 3592.182037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3592.219354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3592.236180] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3597.216689] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3605.718909] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3610.688616] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3618.010347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3618.328305] Lustre: lustre-OST0000: new disk, initializing [ 3618.332254] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3618.342592] Lustre: 121984:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3618.465558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3624.537981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3624.553700] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3624.736065] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3625.045549] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3633.471279] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3638.142858] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3638.427811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3643.435948] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3643.591446] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3650.528694] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3650.534069] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3650.543534] LustreError: Skipped 1 previous similar message [ 3650.544228] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3650.579283] Lustre: Skipped 1 previous similar message [ 3655.652767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3655.663841] Lustre: Skipped 1 previous similar message [ 3660.256269] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3660.367457] Lustre: server umount lustre-OST0000 complete [ 3665.892893] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3665.900026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3665.919659] Lustre: Skipped 1 previous similar message [ 3665.930078] Lustre: Skipped 1 previous similar message [ 3670.418807] Lustre: server umount lustre-MDT0000 complete [ 3673.902121] LustreError: 120234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233365 with bad export cookie 761355991825225390 [ 3673.905825] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3673.909723] LustreError: 120234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3674.245655] Lustre: server umount lustre-MDT0001 complete [ 3682.578388] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 23:02:53 (1781233373) [ 3689.166369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3689.620973] LustreError: 124248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3689.635948] LustreError: 124248:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3689.685259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3693.749632] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3695.073218] LustreError: 124249:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3699.171026] LustreError: 124248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.944705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3707.419799] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3712.400430] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3716.207646] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3724.271654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3724.787105] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3724.792752] Lustre: Skipped 1 previous similar message [ 3732.166788] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3740.011943] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3745.890182] Lustre: *** cfs_fail_loc=159, val=0*** [ 3761.564076] Lustre: lustre-MDT0000: Client a05871b4-caea-40c3-9de2-789c4b6a4d4f (at 192.168.202.12@tcp) reconnecting [ 3763.181232] Lustre: *** cfs_fail_loc=159, val=0*** [ 3763.183510] Lustre: Skipped 5 previous similar messages [ 3778.410724] Lustre: lustre-MDT0000: Client a05871b4-caea-40c3-9de2-789c4b6a4d4f (at 192.168.202.12@tcp) reconnecting [ 3800.559702] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3800.560477] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3800.578753] LustreError: Skipped 1 previous similar message [ 3800.592447] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3800.649574] Lustre: Skipped 1 previous similar message [ 3802.830458] Lustre: server umount lustre-OST0000 complete [ 3810.785736] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3810.791330] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3810.806819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3810.828672] Lustre: Skipped 1 previous similar message [ 3810.847574] Lustre: Skipped 2 previous similar messages [ 3814.604139] Lustre: server umount lustre-MDT0000 complete [ 3815.907297] LustreError: 126448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3815.921980] LustreError: 126448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3818.484310] LustreError: 126631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233510 with bad export cookie 761355991825226272 [ 3818.501328] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3818.511331] LustreError: 126631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3819.031478] Lustre: server umount lustre-MDT0001 complete [ 3830.408773] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 3833.032358] Key type lgssc unregistered [ 3833.340363] LNet: 127671:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3833.349366] LNetError: 127671:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3833.360265] LNet: Removed LNI 192.168.202.112@tcp [ 3834.280682] Key type .llcrypt unregistered [ 3834.284108] Key type ._llcrypt unregistered [ 3861.023578] Key type ._llcrypt registered [ 3861.025945] Key type .llcrypt registered [ 3861.187407] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 3862.357664] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3862.494309] alg: No test for adler32 (adler32-zlib) [ 3863.524168] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 3863.715844] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 3865.351800] Key type lgssc registered [ 3866.341532] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3874.062684] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3874.089974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3875.513556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3880.833346] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3885.028883] LustreError: 129070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3885.054056] LustreError: 129070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3889.868643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3890.287425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3894.685744] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3900.209065] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3903.634435] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3911.151180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3911.407896] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3918.535446] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 3920.643937] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3925.054828] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3934.337658] Lustre: *** cfs_fail_loc=159, val=0*** [ 3950.967612] Lustre: lustre-MDT0000: Client ce7e269b-c50c-4e79-ae7c-c9df50b4c662 (at 192.168.202.12@tcp) reconnecting [ 3954.041346] Lustre: *** cfs_fail_loc=159, val=0*** [ 3969.418910] Lustre: lustre-MDT0001: Client ce7e269b-c50c-4e79-ae7c-c9df50b4c662 (at 192.168.202.12@tcp) reconnecting [ 3971.975135] Lustre: *** cfs_fail_loc=159, val=0*** [ 3971.976928] Lustre: Skipped 3 previous similar messages [ 3988.421215] Lustre: lustre-MDT0001: Client ce7e269b-c50c-4e79-ae7c-c9df50b4c662 (at 192.168.202.12@tcp) reconnecting [ 3992.904866] Lustre: 131770:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3995.757177] Lustre: *** cfs_fail_loc=159, val=0*** [ 3995.759979] Lustre: Skipped 4 previous similar messages [ 4011.432593] Lustre: lustre-MDT0000: Client de95264e-7403-4f33-b2f3-3be53361328f (at 192.168.202.12@tcp) reconnecting [ 4013.095669] Lustre: *** cfs_fail_loc=159, val=0*** [ 4013.101075] Lustre: Skipped 14 previous similar messages [ 4028.769254] Lustre: lustre-MDT0000: Client de95264e-7403-4f33-b2f3-3be53361328f (at 192.168.202.12@tcp) reconnecting [ 4030.420594] Lustre: 132062:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4035.045165] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4035.049654] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4035.053015] LustreError: Skipped 2 previous similar messages [ 4035.072026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4037.096948] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4037.100123] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4037.102810] LustreError: Skipped 6 previous similar messages [ 4037.116618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4038.629881] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4038.643418] Lustre: Skipped 1 previous similar message [ 4043.745363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4043.761888] Lustre: Skipped 1 previous similar message [ 4048.865259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4048.880682] Lustre: Skipped 1 previous similar message [ 4049.376723] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4049.705838] Lustre: server umount lustre-OST0000 complete [ 4059.104576] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4059.106196] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4059.110723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4059.153847] Lustre: Skipped 1 previous similar message [ 4060.612582] Lustre: server umount lustre-MDT0000 complete [ 4064.229206] LustreError: 129070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4064.275967] LustreError: 129070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4064.428542] LustreError: 129051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233756 with bad export cookie 16838630772586605983 [ 4064.435979] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4064.449649] LustreError: 129051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4064.872222] Lustre: server umount lustre-MDT0001 complete [ 4075.302087] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4078.062870] Key type lgssc unregistered [ 4078.344560] LNet: 133099:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4078.352510] LNetError: 133099:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4078.379922] LNet: Removed LNI 192.168.202.112@tcp [ 4079.259829] Key type .llcrypt unregistered [ 4079.265748] Key type ._llcrypt unregistered [ 4107.981111] Key type ._llcrypt registered [ 4107.982830] Key type .llcrypt registered [ 4108.147873] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4109.939054] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4110.130103] alg: No test for adler32 (adler32-zlib) [ 4111.334453] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 4111.597202] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4113.407376] Key type lgssc registered [ 4114.695727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4122.945341] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4122.983182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4124.668650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4129.663044] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4134.884984] LustreError: 134501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4134.906505] LustreError: 134501:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4138.565752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4138.945834] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4143.445232] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4150.365485] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4154.582530] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4162.879584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4163.248048] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4164.279521] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4169.416848] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4169.579845] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4175.315895] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4181.303816] Lustre: 136744:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4184.576330] Lustre: 136793:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4186.370954] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4187.324945] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4192.682828] Lustre: 136949:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4193.913417] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4194.900438] Lustre: 137047:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4200.419798] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4200.429520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4200.432662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4200.453037] Lustre: Skipped 1 previous similar message [ 4204.975292] Lustre: server umount lustre-OST0000 complete [ 4210.672540] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4210.675509] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4210.693677] LustreError: Skipped 1 previous similar message [ 4210.722782] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4210.722790] Lustre: Skipped 1 previous similar message [ 4215.659661] Lustre: server umount lustre-MDT0000 complete [ 4215.779250] LustreError: 134501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4215.799427] LustreError: 134501:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4220.564722] LustreError: 134480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233912 with bad export cookie 14704941088703359607 [ 4220.576027] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4220.584571] LustreError: 134480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4221.463390] Lustre: server umount lustre-MDT0001 complete [ 4233.376591] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4236.226275] Key type lgssc unregistered [ 4236.606671] LNet: 138089:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4236.622851] LNetError: 138089:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4236.655328] LNet: Removed LNI 192.168.202.112@tcp [ 4237.841817] Key type .llcrypt unregistered [ 4237.851829] Key type ._llcrypt unregistered [ 4263.270334] Key type ._llcrypt registered [ 4263.274220] Key type .llcrypt registered [ 4263.404434] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4264.955261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4265.017177] alg: No test for adler32 (adler32-zlib) [ 4265.991392] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 4266.195940] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4267.879246] Key type lgssc registered [ 4268.963652] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4279.198248] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4279.216181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4280.804786] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4285.377750] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4290.020064] LustreError: 139487:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4290.073865] LustreError: 139487:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4294.686916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4295.291330] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4300.461289] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4305.556654] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4309.812150] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4317.001145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4317.315426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4320.388993] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4324.471967] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4325.494202] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4331.035164] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4337.082075] Lustre: *** cfs_fail_loc=159, val=0*** [ 4352.388802] Lustre: lustre-MDT0000: Client 7241464a-70c9-4a49-8486-969e6a811bb4 (at 192.168.202.12@tcp) reconnecting [ 4358.121560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4358.139528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4358.146808] LustreError: Skipped 1 previous similar message [ 4358.194661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4359.732755] Lustre: server umount lustre-OST0000 complete [ 4366.819057] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4366.826799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4366.840506] Lustre: Skipped 1 previous similar message [ 4366.867049] Lustre: Skipped 1 previous similar message [ 4370.402198] Lustre: server umount lustre-MDT0000 complete [ 4371.943462] LustreError: 140200:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4371.960503] LustreError: 140200:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4374.270063] LustreError: 139468:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234066 with bad export cookie 17486646418470030435 [ 4374.274769] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4374.287774] LustreError: 139468:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4384.912416] Lustre: server umount lustre-MDT0001 complete [ 4396.025402] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4398.929967] Key type lgssc unregistered [ 4399.306803] LNet: 142793:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4399.315485] LNetError: 142793:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4399.340311] LNet: Removed LNI 192.168.202.112@tcp [ 4400.456551] Key type .llcrypt unregistered [ 4400.459089] Key type ._llcrypt unregistered [ 4426.570370] Key type ._llcrypt registered [ 4426.572113] Key type .llcrypt registered [ 4426.721127] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4427.833614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4428.004266] alg: No test for adler32 (adler32-zlib) [ 4429.047402] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 4429.326249] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4431.071123] Key type lgssc registered [ 4432.315122] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4441.655802] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4441.675827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4443.156391] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4449.364939] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4453.354280] LustreError: 144193:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4453.401878] LustreError: 144193:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4458.467273] LustreError: 144194:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4459.193849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4459.723717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4465.230981] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4470.811572] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4475.860846] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4483.360492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4483.871149] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4484.933584] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4487.999344] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4490.553247] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4496.814562] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4515.453694] Lustre: lustre-MDT0001: evicting ed7ee473-cc6b-433b-a52e-b8e00a7205e5 (at 192.168.202.12@tcp) by administrative request [ 4515.467812] Lustre: Skipped 1 previous similar message [ 4544.483182] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4544.496077] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4544.500873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4549.600594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4549.622898] Lustre: Skipped 2 previous similar messages [ 4549.926769] Lustre: server umount lustre-OST0000 complete [ 4554.721642] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4554.733312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4554.738287] Lustre: Skipped 2 previous similar messages [ 4559.842201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4559.860776] Lustre: Skipped 2 previous similar messages [ 4560.841540] Lustre: server umount lustre-MDT0000 complete [ 4564.964635] LustreError: 145280:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4564.993616] LustreError: 145280:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4566.513981] LustreError: 144173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234258 with bad export cookie 18085496092813020270 [ 4566.517431] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4566.539924] LustreError: 144173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4566.928932] Lustre: server umount lustre-MDT0001 complete [ 4576.961893] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4580.567411] Key type lgssc unregistered [ 4580.892632] LNet: 147908:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4580.900264] LNetError: 147908:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4580.926677] LNet: Removed LNI 192.168.202.112@tcp [ 4581.866561] Key type .llcrypt unregistered [ 4581.868682] Key type ._llcrypt unregistered [ 4651.076874] Key type ._llcrypt registered [ 4651.080564] Key type .llcrypt registered [ 4651.189791] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4652.202142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4652.243917] alg: No test for adler32 (adler32-zlib) [ 4653.315713] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 4653.564322] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4655.223217] Key type lgssc registered [ 4656.157338] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4662.786942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4662.799886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4664.273388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4668.916404] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4674.529947] LustreError: 150370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4674.552803] LustreError: 150370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4677.121208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4677.530616] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4681.750478] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4686.948466] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4690.226466] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4696.297459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4696.648637] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4702.207317] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4702.831715] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4707.289925] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4708.974837] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4723.683857] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4723.692687] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4723.722094] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.708286] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.717344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4724.722184] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4727.940831] Lustre: server umount lustre-OST0000 complete [ 4733.921050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4733.932903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.957918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4733.961717] Lustre: Skipped 1 previous similar message [ 4738.986919] Lustre: server umount lustre-MDT0000 complete [ 4740.064345] LustreError: 151084:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4740.080600] LustreError: 151084:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4743.106115] LustreError: 150348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234435 with bad export cookie 1763855239236201965 [ 4743.124660] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4743.516185] Lustre: server umount lustre-MDT0001 complete [ 4754.146322] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4757.168877] Key type lgssc unregistered [ 4757.590710] LNet: 153574:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4757.603908] LNetError: 153574:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4758.631656] LNet: Removed LNI 192.168.202.112@tcp [ 4759.767360] Key type .llcrypt unregistered [ 4759.770843] Key type ._llcrypt unregistered [ 4785.221986] Key type ._llcrypt registered [ 4785.223663] Key type .llcrypt registered [ 4785.339374] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4786.463114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4786.722313] alg: No test for adler32 (adler32-zlib) [ 4787.922531] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 4788.218737] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 4789.967208] Key type lgssc registered [ 4791.043723] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4806.380812] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 23:21:37 (1781234497) [ 4824.310280] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 4833.881596] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 4875.466535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4875.511036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4876.863995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4876.920230] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4877.110701] Lustre: lustre-MDT0000: new disk, initializing [ 4877.225861] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4877.250937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4882.404718] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4893.518476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4893.653148] Lustre: 157791:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4893.713711] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4893.718643] Lustre: Skipped 1 previous similar message [ 4893.798360] Lustre: lustre-MDT0001: new disk, initializing [ 4893.930578] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4893.976473] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4893.996548] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4899.004325] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4907.167405] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4911.412278] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4918.838275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4919.121409] Lustre: lustre-OST0000: new disk, initializing [ 4919.125833] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4919.135340] Lustre: 159532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4919.245287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4925.953705] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 4927.573988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4927.597594] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4927.684908] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4934.171163] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4948.457225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4948.457646] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4948.478776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4948.487893] LustreError: Skipped 1 previous similar message [ 4948.507779] Lustre: Skipped 1 previous similar message [ 4951.213452] Lustre: server umount lustre-OST0000 complete [ 4958.689325] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4958.691680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4958.705263] Lustre: Skipped 2 previous similar messages [ 4958.734684] Lustre: Skipped 1 previous similar message [ 4963.151621] Lustre: server umount lustre-MDT0000 complete [ 4968.331176] LustreError: 157783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234660 with bad export cookie 886985479407177845 [ 4968.337288] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4968.345125] LustreError: 157783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4968.660279] Lustre: server umount lustre-MDT0001 complete [ 4981.101294] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 4984.621815] Key type lgssc unregistered [ 4984.972246] LNet: 161315:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4984.975409] LNetError: 161315:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4984.988209] LNet: Removed LNI 192.168.202.112@tcp [ 4986.100111] Key type .llcrypt unregistered [ 4986.103854] Key type ._llcrypt unregistered [ 5012.793525] Key type ._llcrypt registered [ 5012.799976] Key type .llcrypt registered [ 5012.935855] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 5014.276496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5014.379072] alg: No test for adler32 (adler32-zlib) [ 5015.695818] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 5015.965609] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 5017.671239] Key type lgssc registered [ 5019.110698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5028.205609] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5028.247274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5029.977681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5035.051724] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5040.114963] LustreError: 162714:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5040.135352] LustreError: 162714:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5044.051622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5044.491109] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5048.764766] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5054.646603] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5059.117312] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5067.511409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5067.931063] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5075.197476] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5075.498790] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 5082.298082] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5091.059307] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5091.104567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5091.133392] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5091.149823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.173910] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5092.603449] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5092.628989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5092.643487] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5092.649444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.662675] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5093.994094] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5094.043902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5094.053749] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5094.059311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.071579] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5095.326953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.528475] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5096.534255] Lustre: Skipped 1 previous similar message [ 5096.550793] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5096.558453] Lustre: Skipped 1 previous similar message [ 5096.562101] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5096.568831] Lustre: Skipped 1 previous similar message [ 5096.574068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.582117] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5096.592527] Lustre: Skipped 1 previous similar message [ 5097.754838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.069879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.095880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.034641] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5101.037467] Lustre: Skipped 3 previous similar messages [ 5101.049516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5101.059644] Lustre: Skipped 3 previous similar messages [ 5101.061761] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5101.067786] Lustre: Skipped 3 previous similar messages [ 5101.072330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.080241] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5101.086781] Lustre: Skipped 3 previous similar messages [ 5102.209856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.260627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.257260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.655804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.732785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.810191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.584576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.558571] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5109.563825] Lustre: Skipped 7 previous similar messages [ 5109.576315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5109.582902] Lustre: Skipped 7 previous similar messages [ 5109.589882] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5109.592975] Lustre: Skipped 7 previous similar messages [ 5109.594716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.606028] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5109.609096] Lustre: Skipped 7 previous similar messages [ 5110.415194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.389591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.259753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.387614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.153887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.022828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.830027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.706743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.618545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.650284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.473766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.496257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.381541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.358716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.097476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.043790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.792150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.714517] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5125.720986] Lustre: Skipped 17 previous similar messages [ 5125.746843] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5125.767529] Lustre: Skipped 17 previous similar messages [ 5125.773638] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5125.776874] Lustre: Skipped 17 previous similar messages [ 5125.780011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.796696] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5125.818528] Lustre: Skipped 17 previous similar messages [ 5126.727682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.528643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.193836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.440576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.262202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.228029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.085523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.204525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.088177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.048931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.005348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.193375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.198651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.109305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.167917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.037986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.958625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.181912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.458287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.405938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.250570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.053405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.059990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.972531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.094487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.077309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.100389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.008340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.900403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.993659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.027311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.993987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.938225] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5157.945804] Lustre: Skipped 32 previous similar messages [ 5157.963618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5157.981330] Lustre: Skipped 32 previous similar messages [ 5157.987385] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5158.002655] Lustre: Skipped 32 previous similar messages [ 5158.011979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.026855] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5158.032282] Lustre: Skipped 32 previous similar messages [ 5159.264520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.336311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.384324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.380223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.306583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.242074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.028755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.132501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.243752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.358622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.427834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.180183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.036414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.989458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.946329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.036484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.949375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.178612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.172865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.251762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.336791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.227964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.353911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.137545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.225966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.227749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.367168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.225735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.404554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.415156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.451090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.679701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.859056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.914778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.828365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.088655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.087367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.063921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.981557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.953910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.993778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.833429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.646695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.549576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.367629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.507978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.524992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.403326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.240282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.298547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.064276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.873885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.731917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.638303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.408566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.364520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.242248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.412196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.420287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.427978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.437780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.573195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.644517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.753665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.537830] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5222.543449] Lustre: Skipped 64 previous similar messages [ 5222.569705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5222.585928] Lustre: Skipped 64 previous similar messages [ 5222.595081] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5222.602843] Lustre: Skipped 64 previous similar messages [ 5222.617976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.633732] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5222.645325] Lustre: Skipped 64 previous similar messages [ 5223.601625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.379751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.208473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.108536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.111081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.843185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.764294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.707681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.654966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.605258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.766609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.798969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.884031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.002416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.307773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.654818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.739591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.596428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.544438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.420339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.347150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.183673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.556685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.687511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.790355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.746588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.885364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.915978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.003111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.824813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.724730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.563652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.461752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.828399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.035703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.165925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.148296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.253556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.027961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.922534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.964530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.054078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.081208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.224026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.105903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.985489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.031221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.797711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.930389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.935487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.985799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.029762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.170040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.014278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.096552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.030183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.098588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.936720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.665382] LustreError: 162734:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5281.671798] LustreError: 162734:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5281.685957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.538279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.436527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.293046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.202521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.209142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.191818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.044620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.101700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.110695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.291657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.284781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.275929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.135777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.174348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.911567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.840483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.855247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.880017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.870856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.006691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.962168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.047342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.017946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.172922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.109482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.389240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.429211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.462906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.704430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.854712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.045565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.993440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.413097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.516860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.512795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.662676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.998990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.423589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.619946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.711705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.957665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.944618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.126279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.335060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.429595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.607583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.605490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.844576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.142580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.215657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.179210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.949239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.982977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.875554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.875872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.626433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.480479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.165445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.070390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.849281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.656028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.476601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.253812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.137415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.987819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.877246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.898790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.791850] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5350.802784] Lustre: Skipped 126 previous similar messages [ 5350.835995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5350.858743] Lustre: Skipped 126 previous similar messages [ 5350.875282] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5350.884931] Lustre: Skipped 126 previous similar messages [ 5350.888405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.910437] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5350.923981] Lustre: Skipped 126 previous similar messages [ 5352.131613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.009872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.064755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.819645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.623411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.518943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.386652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.580074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.495569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.699613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.689345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.728370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.747504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.657802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.838397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.133375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.978637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.025656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.875624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.756213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.660632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.659387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.504820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.248847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.040854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.899965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.778307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.784795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.894112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.795348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.962538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.934514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.227516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.409359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.589167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.756222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.159747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.231626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.067824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.015860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.967561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.925552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.676579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.679289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.679717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.566571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.551934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.631995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.651537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.731451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.673382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.731680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.781944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.739680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.674228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.546984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.382907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.459577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.270987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.277049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.127265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.989930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.789820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.690863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.551237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.410349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.380083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.471150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.424375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.545405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.464089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.451427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.280431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.366578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.219208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.378139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.301320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.506679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.668140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.747060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.133930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.528827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.578871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.747071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.300877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.366081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.744888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.898180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.140429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.374483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.898399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.205602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.478248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.482897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.398316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.132645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.122585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.240703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.044564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.978844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.138164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.177895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.022322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.461996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.761111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.896337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.798560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.680250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.452741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.240900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.215516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.281074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.648720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.924291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.130684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.270845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.344337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.490089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.674918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.689184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.843134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.233679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.217362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.151900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.961257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.900916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.593960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.417073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.320829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.248413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.049294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.836338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.657996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.630199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.556247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.478947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.420536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.674769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.654380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.511262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.527303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.491236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.404880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.299636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.441703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.386650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.299680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.969050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.875537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.810802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.605747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.473787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.427986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.321816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.310273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.267981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.054391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.946076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.913082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.841552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.865826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.789756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.769819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.845495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.651811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.506140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.411238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.225454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.972149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.049060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.201064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.212603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.251472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.272899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.216546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.969320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.859544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.626449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.592292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.588499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.651267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.446602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.338637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.263738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.294988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.479274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.542297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.407465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.318978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.525614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.701555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.720146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.269269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.161790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.009849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.288897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.334402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.325634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.425313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.593380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.255478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.239596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.117868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.026097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.122346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.288853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.653455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.719110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.215336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.415534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.433100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.163937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.957655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.110778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.176656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.404134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.536905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.512089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.521239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.329730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.315284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.269796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.257692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.083143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.122305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.199992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.158453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.220323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.060349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.998756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.862065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.769720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.678361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.541921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.337631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.374880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.124545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.213722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.094972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.998809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.863888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.804565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.746039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.826134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.850980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.804438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.781645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.631476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.504032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.627545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.525473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.479919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.377631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.398768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.362192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.445410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.357769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.230071] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5607.239564] Lustre: Skipped 257 previous similar messages [ 5607.263458] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5607.280484] Lustre: Skipped 257 previous similar messages [ 5607.293969] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5607.306510] Lustre: Skipped 257 previous similar messages [ 5607.309837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.324195] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5607.336984] Lustre: Skipped 257 previous similar messages [ 5608.303687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.797216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.884274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.744624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.896401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.812390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.849922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.779441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.794389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.798741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.100768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.388645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.206932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.270538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.113761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.799494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.224754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.325389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.330750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.486937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.458629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.760491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.938752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.852444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.531051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.366841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.184802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.072548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.960514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.086987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.907337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.829900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.652287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.434663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.191622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.123441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.047186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.898037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.796875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.692565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.561781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.435567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.344650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.165201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.240448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.265972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.281927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.191659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.999952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.945356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.938083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.870268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.921652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.992426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.039247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.926056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.927661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.810320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.844269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.598458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.453938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.446502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.681417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.541130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.595902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.584176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.501638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.384565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.274696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.227209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.285740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.282299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.315753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.211122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.204156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.389135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.351066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.475852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.390063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.383978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.564217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.489376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.489941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.543832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.581798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.697100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.848314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.119976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.210236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.142987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.128992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.307566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.999951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.543284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.794865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.953678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.068093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.225470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.256980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.458693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.398354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.305239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.566624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.687098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.765954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.675498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.811991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.959440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.174861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.415839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.542718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.665907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.555541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.697561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.067080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.156692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.222503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.513746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.484169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.460592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.190941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.184812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.970522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.074860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.198828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.387457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.450434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.464344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.454889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.326537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.216709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.478348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.703476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.944220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.904953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.994940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.854057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.572666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.309878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.452504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.382282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.546502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.778152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.902677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.238340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.194671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.234487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.266721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.203056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.051453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.215056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.179265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.120158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.996145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.081532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.923633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.995111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.050477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.262295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.289710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.555258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.485219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.686750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.955319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.877951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.955180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.992213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.905741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.962441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.843758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.713575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.732984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.820440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.760829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.863359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.807202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.779355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.726280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.614283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.309836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.009390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.965231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.956924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.778468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.850049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.739232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.709217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.665253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.619693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.474713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.376422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.217421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.137538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.020480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.751891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.297130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.051691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.798201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.566342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.580903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.512637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.704979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.657618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.485813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.594164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.539181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.829685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.139736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.149579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.436438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.527107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.323628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.321605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.369268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.408952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.376341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.570928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.430864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.379161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.049559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.027847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.048077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.244857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.116611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.546994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.946415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.454804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.463852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.420290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.384870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.508051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.475181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.598662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.527886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.541957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.470232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.273651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.144403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.347608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.732618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.747090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.806395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.732337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.827664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.583123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.293231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.463204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.719638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.574123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.824369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.853982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.979518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.716935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.380335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.960232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.759107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.399215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.410801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.290682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.282480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.255083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.211400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.095943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.924163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.746596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.834737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.668285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.501723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.425940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.323396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.159852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.927638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.768867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.699492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.675533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.711187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.665680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.648743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.579972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.548398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.400432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.342653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.342059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.170463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.126201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.155878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.233375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.186924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.035569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.860210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.685735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.603743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.800263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.553631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.513406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.511195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.531139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.561541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.573529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.466267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.372683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.354700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.310877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.092693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.121723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.921514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.737770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.499785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.325269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.306046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.342884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.177112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.191174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.065189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.932173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.669991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.554929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.437830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.325518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.064232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.901633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.677853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.503081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.456794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.304186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.153348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.102469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.978953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.880987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.719581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.636868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.467927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.529215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.453553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.475500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.250395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.171529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.046390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.010975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.943197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.832290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.751616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.818548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.842905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.706823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.491242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.345053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.101549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.233930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.154689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.128349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.095611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.972571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.000327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.109302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.466794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.711896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.997220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.155846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.336711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.483849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.484528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.413596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.347429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.356068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.154914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.027735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.801338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.740965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.602565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.532574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.492409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.529655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.359430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.259739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.118900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.011405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.829301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.763270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.479389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.276709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.067164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.937840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.694752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.561178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.499107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.586134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.631819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.499471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.339694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.300840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.993648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.865907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.633925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.558772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.430203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.470599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.413401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.329693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.244524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.237615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.095774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.780705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.344322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.216277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.276373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.034484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.904835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.735853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.879051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.866328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.078733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.204653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.444806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.793614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.755088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.666587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.483209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.365840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.162201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.844226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.427080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.308066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.369229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.370693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.716825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.645431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.624582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.861991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.988040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.040770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.042971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.282314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.529597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.816283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.794937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.883688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.696477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.790686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.075985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.156927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.184527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.180385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.143537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.235980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.384272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.765902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.705429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.905362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.867898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.110992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.055708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.047275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.919507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.968748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.764316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.803204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.820930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.634730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.716986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.746885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.759622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.733256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.591551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.597710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.589853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.603945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.866105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.669199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.774587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.745377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.811364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.953341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.819992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.901445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.777516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.638237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.584092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.507929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.341094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.361297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.179598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.464890] LustreError: 165684:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6408.484445] LustreError: 162734:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6847.972642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6847.989628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6847.993549] Lustre: Skipped 483 previous similar messages [ 6848.030836] Lustre: Skipped 1 previous similar message [ 6848.996745] Lustre: server umount lustre-OST0000 complete [ 6853.097830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6853.109844] Lustre: Skipped 1 previous similar message [ 6858.212487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6858.222121] Lustre: Skipped 1 previous similar message [ 6859.530696] Lustre: server umount lustre-MDT0000 complete [ 6863.333103] LustreError: 162710:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6863.365219] LustreError: 162710:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6863.376908] LustreError: 191164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236555 with bad export cookie 6640501414032402759 [ 6863.380299] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6863.398198] LustreError: 191164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6863.834794] Lustre: server umount lustre-MDT0001 complete [ 6875.632338] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 6878.447127] Key type lgssc unregistered [ 6878.829726] LNet: 217577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6878.840146] LNetError: 217577:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6878.870288] LNet: Removed LNI 192.168.202.112@tcp [ 6879.751950] Key type .llcrypt unregistered [ 6879.754275] Key type ._llcrypt unregistered [ 6905.110390] Key type ._llcrypt registered [ 6905.112618] Key type .llcrypt registered [ 6905.317380] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 6906.346732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6906.455437] alg: No test for adler32 (adler32-zlib) [ 6908.157511] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 6908.640071] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 6910.399132] Key type lgssc registered [ 6912.210850] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6921.778962] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6921.807877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6923.591429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6929.411093] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6932.960159] LustreError: 218993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6932.970609] LustreError: 218993:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6938.088675] LustreError: 218992:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6938.677585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6939.209827] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6943.322616] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6949.631704] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6953.425137] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6961.042959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6961.346755] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6967.398784] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 6968.506079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6973.130506] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6977.994368] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6979.786742] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6979.808867] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6979.816846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.835792] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6979.842167] LustreError: 219012:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6984.163348] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6984.167425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6984.192124] Lustre: Skipped 1 previous similar message [ 6984.209802] Lustre: Skipped 1 previous similar message [ 6985.045720] Lustre: lustre-OST0000: Not available for connect from 192.168.202.12@tcp (stopping) [ 6988.317414] Lustre: server umount lustre-OST0000 complete [ 6999.911091] LustreError: 219012:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7023.948512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7024.111379] LustreError: 221863:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7024.285894] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7029.747518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.762353] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.763347] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7029.923640] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7035.050572] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7036.642251] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7040.579086] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c74c32c8000.ost_server_uuid 50 [ 7042.359132] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c74c32c8000.ost_server_uuid in FULL state after 0 sec [ 7047.647879] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7047.655264] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7047.679229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7048.664853] Lustre: server umount lustre-OST0000 complete [ 7055.329802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7055.331758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7055.354026] Lustre: Skipped 1 previous similar message [ 7055.376031] Lustre: Skipped 1 previous similar message [ 7058.223955] Lustre: server umount lustre-MDT0000 complete [ 7060.448878] LustreError: 218989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7060.470961] LustreError: 218989:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7061.964668] LustreError: 218972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236754 with bad export cookie 8570049774432242481 [ 7061.966821] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7061.978940] LustreError: 218972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7072.425680] Lustre: server umount lustre-MDT0001 complete [ 7084.511326] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 7086.802730] Key type lgssc unregistered [ 7087.033935] LNet: 223637:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7087.041829] LNetError: 223637:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7088.101059] LNet: Removed LNI 192.168.202.112@tcp [ 7089.080470] Key type .llcrypt unregistered [ 7089.083273] Key type ._llcrypt unregistered [ 7120.351796] Key type ._llcrypt registered [ 7120.356456] Key type .llcrypt registered [ 7120.462928] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7122.032668] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7122.366957] alg: No test for adler32 (adler32-zlib) [ 7123.558309] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 7123.721781] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 7125.367175] Key type lgssc registered [ 7126.518235] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7136.538674] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7136.562850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7138.151111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7142.380918] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7148.530978] LustreError: 225352:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7148.573747] LustreError: 225352:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7151.086426] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7151.412240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7155.619195] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7158.591413] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7164.919408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7165.230368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7171.750364] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7180.281338] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7180.485803] Lustre: lustre-OST0001: new disk, initializing [ 7180.489132] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7180.494425] Lustre: 227623:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7180.587549] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7183.673707] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7186.526814] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7188.688583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7188.701369] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7188.796302] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7198.614480] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7206.954349] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7228.384532] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7228.398122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7228.414577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7229.921959] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7229.927053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7229.930444] Lustre: Skipped 2 previous similar messages [ 7229.939666] Lustre: Skipped 3 previous similar messages [ 7233.804338] Lustre: server umount lustre-MDT0000 complete [ 7235.049904] LustreError: 225347:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7235.078308] LustreError: 225347:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7237.667891] LustreError: 228381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236929 with bad export cookie 12312864278931579789 [ 7237.683751] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7237.689660] LustreError: 228381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7238.214843] Lustre: server umount lustre-MDT0001 complete [ 7252.660862] Lustre: server umount lustre-OST0000 complete [ 7256.353085] Lustre: 224616:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781236932/real 1781236932] req@ffff8a2b155cb480 x1867762174638080/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781236948 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7256.387927] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7256.783151] Lustre: server umount lustre-OST0001 complete [ 7280.933901] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7291.501721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7291.956987] LustreError: 230704:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7292.079854] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7295.755798] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7297.506317] LustreError: 230705:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7297.526204] LustreError: 230705:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7302.630900] LustreError: 230704:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7303.144867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7303.647787] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7307.733497] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7310.743302] Lustre: 231812:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7317.092562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7323.984929] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7328.767834] LustreError: 232166:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7328.793089] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7333.985576] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7334.363230] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7334.370491] Lustre: Skipped 1 previous similar message [ 7337.478266] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7340.394322] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7348.016473] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7351.704126] Lustre: 233649:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7363.043345] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7363.051573] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7363.064649] Lustre: Skipped 2 previous similar messages [ 7363.084598] Lustre: Skipped 3 previous similar messages [ 7368.190048] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7368.213852] Lustre: Skipped 3 previous similar messages [ 7373.287120] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7376.351154] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7376.557262] Lustre: server umount mylustre-MDT0000 complete [ 7378.406251] LustreError: 230701:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7378.426382] LustreError: 230701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7380.481163] LustreError: 230681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237072 with bad export cookie 12312864278931582659 [ 7380.486523] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7380.491693] LustreError: 230681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7380.783821] Lustre: server umount mylustre-MDT0001 complete [ 7394.868342] Lustre: server umount mylustre-OST0000 complete [ 7409.315872] Lustre: server umount mylustre-OST0001 complete [ 7430.635632] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7442.043033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7442.612254] LustreError: 235944:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7442.733177] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7446.902858] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7455.523621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7460.654381] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7463.596442] Lustre: 237053:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7471.273272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7477.390266] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7479.992549] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7485.544638] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7487.889823] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7492.666790] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7501.674497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7506.362460] Lustre: 238891:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7517.664826] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7517.686359] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7517.701205] Lustre: Skipped 1 previous similar message [ 7517.719488] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7517.734839] Lustre: Skipped 3 previous similar messages [ 7521.162736] Lustre: server umount tfs-MDT0000 complete [ 7523.815167] LustreError: 236660:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7523.828506] LustreError: 236660:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 7525.029165] LustreError: 238894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237217 with bad export cookie 12312864278931584612 [ 7525.039447] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7525.043148] LustreError: 238894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7525.416128] Lustre: server umount tfs-MDT0001 complete [ 7540.107654] Lustre: server umount tfs-OST0000 complete [ 7555.857472] Lustre: server umount tfs-OST0001 complete [ 7580.128969] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7591.240292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7591.708665] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7591.717308] Lustre: Skipped 3 previous similar messages [ 7596.875614] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7608.842677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7614.870955] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7617.646885] Lustre: 242237:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7623.555846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7630.756715] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7634.264523] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7636.320627] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7640.362571] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7647.778777] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7657.329353] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7662.878640] Lustre: 244074:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7671.171555] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 00:09:21 (1781237361) [ 7691.744366] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7691.754667] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7691.768901] Lustre: Skipped 3 previous similar messages [ 7691.772099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7691.776577] Lustre: Skipped 4 previous similar messages [ 7694.886991] Lustre: server umount lustre-MDT0000 complete [ 7697.390430] LustreError: 241125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7697.426720] LustreError: 241125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 7698.588204] LustreError: 241103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237390 with bad export cookie 12312864278931586299 [ 7698.594330] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7698.604772] LustreError: 241103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7705.013213] Lustre: server umount lustre-MDT0001 complete [ 7714.801982] Lustre: server umount lustre-OST0000 complete [ 7733.215516] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7733.494505] Lustre: server umount lustre-OST0001 complete [ 7740.857482] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 7750.623951] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 7785.504553] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7785.504553] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7785.504553] [ 7785.563920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7785.794375] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7785.832602] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7785.908803] Lustre: lustre-MDT0000: new disk, initializing [ 7785.983312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7785.986434] Lustre: Skipped 3 previous similar messages [ 7786.008382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7790.543775] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7801.039945] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7801.039945] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7801.039945] [ 7801.067754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7801.213492] Lustre: 247229:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7801.275799] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7801.280470] Lustre: Skipped 1 previous similar message [ 7801.424656] Lustre: lustre-MDT0001: new disk, initializing [ 7801.551040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7801.567260] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7806.070089] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7816.583453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7816.808236] Lustre: lustre-OST0000: new disk, initializing [ 7816.811452] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7816.825621] Lustre: 248776:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7818.240259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7818.258651] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7818.370304] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7823.994936] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7836.294633] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7836.471603] Lustre: lustre-OST0001: new disk, initializing [ 7836.478720] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7836.489877] Lustre: 249629:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7838.193993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7838.216779] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7838.360260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7843.168496] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 7854.610919] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 00:12:26 (1781237546) [ 7873.492339] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 00:12:43 (1781237563) [ 7889.395089] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7889.405562] Lustre: Skipped 5 previous similar messages [ 7889.409153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7889.417225] Lustre: Skipped 9 previous similar messages [ 7895.391189] Lustre: server umount lustre-OST0000 complete [ 7904.236851] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7906.575067] Lustre: server umount lustre-MDT0000 complete [ 7911.500133] LustreError: 247219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237603 with bad export cookie 12312864278931587860 [ 7911.505874] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7911.522571] LustreError: 247219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7918.142937] Lustre: server umount lustre-MDT0001 complete [ 7931.701141] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 7942.963785] Lustre: server umount lustre-OST0001 complete [ 7950.473468] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 7966.757767] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8006.100079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8006.388351] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8006.448281] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8006.568580] Lustre: lustre-MDT0000: new disk, initializing [ 8006.751932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8011.610691] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8023.704552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8023.917349] Lustre: 254120:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8024.057317] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8024.071066] Lustre: Skipped 1 previous similar message [ 8024.276708] Lustre: lustre-MDT0001: new disk, initializing [ 8024.555408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8024.567945] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8030.050156] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8038.393412] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8042.429457] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8049.226163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8049.482507] Lustre: 255864:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8049.603330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8049.606904] Lustre: Skipped 5 previous similar messages [ 8051.601194] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8051.728880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8055.549513] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8063.320987] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8072.159994] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8072.170870] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8072.191397] LustreError: Skipped 1 previous similar message [ 8072.194154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8072.212580] Lustre: Skipped 5 previous similar messages [ 8072.239024] Lustre: Skipped 8 previous similar messages [ 8076.866405] Lustre: server umount lustre-OST0000 complete [ 8087.513957] Lustre: server umount lustre-MDT0000 complete [ 8087.528209] LustreError: 254133:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8087.550404] LustreError: 254133:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 8091.373879] LustreError: 255863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237783 with bad export cookie 12312864278931589330 [ 8091.376970] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8091.381748] LustreError: 255863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8091.731146] Lustre: server umount lustre-MDT0001 complete [ 8104.286628] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 8107.590709] Key type lgssc unregistered [ 8107.848793] LNet: 257494:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8107.865713] LNetError: 257494:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8108.907052] LNet: Removed LNI 192.168.202.112@tcp [ 8109.982692] Key type .llcrypt unregistered [ 8109.985428] Key type ._llcrypt unregistered [ 8137.299681] Key type ._llcrypt registered [ 8137.301810] Key type .llcrypt registered [ 8137.418990] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8138.893347] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8138.939167] alg: No test for adler32 (adler32-zlib) [ 8140.057680] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 8140.281805] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 8141.989904] Key type lgssc registered [ 8143.310647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8151.354696] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8151.383565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8153.110309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8157.981921] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8163.302766] LustreError: 258956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8163.311378] LustreError: 258956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8167.048899] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8167.542631] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8171.480640] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8177.734643] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8182.547876] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8190.115133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8190.464766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8196.830289] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8198.655232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8202.729214] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8203.837644] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8207.848903] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8207.858527] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8207.870748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8208.864917] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8208.875544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8208.875937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8208.910105] Lustre: Skipped 1 previous similar message [ 8211.886329] Lustre: server umount lustre-OST0000 complete [ 8218.592863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8218.598761] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8218.612498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8224.226468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8224.237526] Lustre: Skipped 3 previous similar messages [ 8229.346470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8229.364714] Lustre: Skipped 1 previous similar message [ 8230.367206] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8230.641782] Lustre: server umount lustre-MDT0000 complete [ 8234.482782] LustreError: 258957:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8234.523040] LustreError: 258957:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8234.555860] LustreError: 258936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237926 with bad export cookie 10545641753832517548 [ 8234.565352] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8234.575592] LustreError: 258936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8235.209740] Lustre: server umount lustre-MDT0001 complete [ 8249.300852] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 8253.302411] Key type lgssc unregistered [ 8253.631813] LNet: 262188:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8253.639415] LNetError: 262188:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8253.657508] LNet: Removed LNI 192.168.202.112@tcp [ 8254.914918] Key type .llcrypt unregistered [ 8254.919381] Key type ._llcrypt unregistered [ 8275.779447] Key type ._llcrypt registered [ 8275.782595] Key type .llcrypt registered [ 8275.979988] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8277.724618] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8277.750578] alg: No test for adler32 (adler32-zlib) [ 8278.925716] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 8279.128984] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 8280.871192] Key type lgssc registered [ 8282.287051] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8293.579961] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8293.631243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8309.343367] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8314.463244] LustreError: 263371:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.112@tcp: failed processing log, type 4: rc = -110 [ 8341.219804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8347.972224] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8355.017684] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8356.891908] Lustre: server umount lustre-OST0000 complete [ 8372.763266] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 8375.324679] Key type lgssc unregistered [ 8375.703048] LNet: 264871:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8375.715645] LNetError: 264871:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8375.734891] LNet: Removed LNI 192.168.202.112@tcp [ 8376.408532] Key type .llcrypt unregistered [ 8376.410949] Key type ._llcrypt unregistered [ 8411.576834] Key type ._llcrypt registered [ 8411.583680] Key type .llcrypt registered [ 8411.690295] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8412.670921] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8412.695109] alg: No test for adler32 (adler32-zlib) [ 8413.821800] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 8414.135802] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 8415.807176] Key type lgssc registered [ 8417.064366] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8430.506901] loop0: detected capacity change from 0 to 204800000 [ 8430.777191] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8430.804212] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8431.170304] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.243220] loop0: detected capacity change from 0 to 204800000 [ 8432.451658] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.470827] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8432.892060] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8433.905971] loop0: detected capacity change from 0 to 204800000 [ 8434.093400] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8434.099803] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8434.216813] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8435.279347] loop0: detected capacity change from 0 to 204800000 [ 8435.506601] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8436.650172] loop0: detected capacity change from 0 to 204800000 [ 8436.675154] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8440.837084] loop0: detected capacity change from 0 to 204800000 [ 8440.896209] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8444.856519] loop0: detected capacity change from 0 to 204800000 [ 8444.892897] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8449.168342] loop0: detected capacity change from 0 to 204800000 [ 8449.187649] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8453.109660] loop0: detected capacity change from 0 to 204800000 [ 8453.226625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8453.299841] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8453.355541] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8453.366481] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8453.381993] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8453.395883] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8453.409668] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8453.433974] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8453.449319] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8453.473806] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8453.499431] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8453.518198] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8453.537954] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8453.547842] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8453.557646] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8453.571472] Lustre: 268264:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8459.757620] Lustre: server umount MGS complete [ 8460.678690] loop0: detected capacity change from 0 to 204800000 [ 8460.868398] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8461.327063] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8461.397962] Lustre: lustre-MDT0000: Aborting client recovery [ 8462.309603] loop1: detected capacity change from 0 to 204800000 [ 8462.530168] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8462.602392] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8462.611233] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8462.621753] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8462.630673] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8462.639213] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8462.647817] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8462.664396] Lustre: 268696:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8462.712108] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8462.992930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8463.092554] Lustre: lustre-MDT0001: Aborting client recovery [ 8464.017757] loop2: detected capacity change from 0 to 204800000 [ 8464.216787] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8464.347046] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8464.378849] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8464.398265] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8464.426578] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8464.455505] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8464.473479] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8464.487600] Lustre: 268775:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8464.592342] Lustre: Found index 0 for lustre-OST0000, updating log [ 8464.645983] Lustre: 268793:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8464.732157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8464.767955] Lustre: lustre-OST0000: Aborting recovery [ 8465.736668] loop3: detected capacity change from 0 to 204800000 [ 8465.903716] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8466.010116] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8466.034812] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8466.047575] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8466.059042] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8466.067599] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8466.088788] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8466.102092] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8466.128610] Lustre: Found index 1 for lustre-OST0001, updating log [ 8466.151122] Lustre: 268864:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8466.240264] Lustre: lustre-OST0001: Aborting recovery [ 8471.340994] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8471.351844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8471.368579] LustreError: 268640:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8471.379603] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8471.401053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8472.361803] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8472.374870] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8475.217981] LustreError: 268607:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4514us: rc = -2 [ 8475.769805] LustreError: 268608:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 415us: rc = -2 [ 8475.779969] LustreError: 268608:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8487.904773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8487.925392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8487.926753] Lustre: Skipped 2 previous similar messages [ 8487.942733] Lustre: Skipped 3 previous similar messages [ 8493.036857] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8498.155640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8498.166506] Lustre: Skipped 6 previous similar messages [ 8502.239584] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8502.495803] Lustre: server umount lustre-MDT0000 complete [ 8503.265628] LustreError: 268609:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8503.285543] LustreError: 268609:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8503.493476] LustreError: 268591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238195 with bad export cookie 8284905069748706555 [ 8503.505839] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8503.514412] LustreError: 268591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8508.383900] LustreError: 268606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8508.385543] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8508.388435] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8508.415823] LustreError: 268606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8508.468839] Lustre: Skipped 2 previous similar messages [ 8510.139169] Lustre: server umount lustre-MDT0001 complete [ 8517.276533] Lustre: server umount lustre-OST0000 complete [ 8524.676246] Lustre: server umount lustre-OST0001 complete [ 8532.435488] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 00:23:43 (1781238223) [ 8561.249856] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 8570.206708] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8608.076719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8608.321965] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8608.353896] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8608.420959] Lustre: lustre-MDT0000: new disk, initializing [ 8608.486507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8608.494796] Lustre: Skipped 1 previous similar message [ 8608.506420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8613.324968] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8625.291529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.501566] Lustre: 272916:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8625.568091] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8625.572874] Lustre: Skipped 1 previous similar message [ 8625.713381] Lustre: lustre-MDT0001: new disk, initializing [ 8625.864514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8625.960181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8625.969773] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8631.030481] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8638.736896] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8643.251589] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8652.552625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8652.996301] Lustre: lustre-OST0000: new disk, initializing [ 8653.004448] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8653.015994] Lustre: 274655:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8653.163317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8659.655729] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8661.638695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8661.651677] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8661.759444] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8666.802476] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8672.246951] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8683.864525] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8690.275839] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8743.907346] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8743.917866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8743.927664] Lustre: Skipped 1 previous similar message [ 8743.957567] Lustre: Skipped 2 previous similar messages [ 8745.928902] Lustre: server umount lustre-OST0000 complete [ 8754.148439] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8754.153278] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8754.171974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8754.181095] Lustre: Skipped 1 previous similar message [ 8756.980081] Lustre: server umount lustre-MDT0000 complete [ 8759.265150] LustreError: 274304:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8759.293567] LustreError: 274304:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8761.919169] LustreError: 274664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238453 with bad export cookie 8284905069748709264 [ 8761.920516] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8761.937940] LustreError: 274664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8762.358717] Lustre: server umount lustre-MDT0001 complete [ 8773.427121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8778.502675] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8783.428817] Lustre: server umount MGS complete [ 8796.165106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8796.963876] LustreError: 278392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8797.003177] LustreError: 278405:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8797.207471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8802.274478] LustreError: 278393:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8802.292873] LustreError: 278393:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8804.062143] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8812.523714] LustreError: 278392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8812.553856] LustreError: 278392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8814.637677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8819.889525] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8825.334551] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8829.674381] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8837.096402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8837.505101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8837.508524] Lustre: Skipped 1 previous similar message [ 8844.282379] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 8850.120600] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8876.518966] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8876.531242] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8876.548362] Lustre: Skipped 1 previous similar message [ 8876.563460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8878.570274] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8879.957674] Lustre: server umount lustre-OST0000 complete [ 8889.781767] Lustre: server umount lustre-MDT0000 complete [ 8893.037224] LustreError: 278372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238585 with bad export cookie 8284905069748711105 [ 8893.042996] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8893.045774] LustreError: 278372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8893.273933] Lustre: server umount lustre-MDT0001 complete [ 8903.256242] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 8906.156464] Key type lgssc unregistered [ 8906.518298] LNet: 281982:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8906.536055] LNetError: 281982:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8906.561378] LNet: Removed LNI 192.168.202.112@tcp [ 8907.547818] Key type .llcrypt unregistered [ 8907.550423] Key type ._llcrypt unregistered [ 8944.413259] Key type ._llcrypt registered [ 8944.415612] Key type .llcrypt registered [ 8944.530630] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 8958.755806] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 8960.070959] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8960.289990] alg: No test for adler32 (adler32-zlib) [ 8961.367693] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 8961.644664] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 8963.367197] Key type lgssc registered [ 8964.595394] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9003.769115] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9003.803415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9005.175882] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9005.239762] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9005.298123] Lustre: lustre-MDT0000: new disk, initializing [ 9005.371100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9005.402063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9010.522872] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9023.993959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9024.101422] Lustre: 285905:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9024.149467] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9024.157191] Lustre: Skipped 1 previous similar message [ 9024.248773] Lustre: lustre-MDT0001: new disk, initializing [ 9024.387973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9024.463463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9024.475451] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9029.665878] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9038.519624] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9042.989531] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9050.540897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9051.005812] Lustre: lustre-OST0000: new disk, initializing [ 9051.015810] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9051.024655] Lustre: 287647:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9051.140550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9058.769047] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9061.472734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9061.498947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9061.594520] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9068.352576] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9076.035374] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9087.360167] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9093.959365] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9106.184571] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9148.896778] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9148.904515] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9148.906481] LustreError: Skipped 1 previous similar message [ 9148.915751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9148.921302] Lustre: Skipped 1 previous similar message [ 9148.946255] Lustre: Skipped 1 previous similar message [ 9150.630367] Lustre: server umount lustre-OST0000 complete [ 9157.601150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9157.616627] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9157.639661] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9159.136294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9159.136390] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9161.157179] Lustre: server umount lustre-MDT0000 complete [ 9165.050095] LustreError: 288391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238857 with bad export cookie 16324160107156157744 [ 9165.055693] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9165.063363] LustreError: 288391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9165.778343] Lustre: server umount lustre-MDT0001 complete [ 9174.875651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9180.039608] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9185.350967] Lustre: server umount MGS complete [ 9199.026420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9199.378590] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9199.397533] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9199.399839] LustreError: 291420:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9199.466049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9203.791572] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9204.715525] LustreError: 291408:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9209.826171] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9214.030444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9214.589354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9219.627779] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9226.112741] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9230.028335] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9236.807668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9237.100756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9244.020695] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9250.722955] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9280.481215] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9280.488864] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9280.527037] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9280.530511] Lustre: Skipped 1 previous similar message [ 9285.601152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9285.618130] Lustre: Skipped 1 previous similar message [ 9294.303292] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9294.579904] Lustre: server umount lustre-OST0000 complete [ 9300.967350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9300.976793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9300.985556] Lustre: Skipped 2 previous similar messages [ 9301.000277] Lustre: Skipped 3 previous similar messages [ 9313.250447] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9313.728250] Lustre: server umount lustre-MDT0000 complete [ 9316.325672] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9316.370142] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9318.663347] LustreError: 291389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781239010 with bad export cookie 16324160107156159599 [ 9318.667326] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9318.671979] LustreError: 291389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9319.220531] Lustre: server umount lustre-MDT0001 complete [ 9332.700644] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [ 9336.792395] Key type lgssc unregistered [ 9337.277575] LNet: 294998:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9337.289775] LNetError: 294998:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9337.323703] LNet: Removed LNI 192.168.202.112@tcp [ 9339.002645] Key type .llcrypt unregistered [ 9339.006532] Key type ._llcrypt unregistered [ 9362.669621] Key type ._llcrypt registered [ 9362.671737] Key type .llcrypt registered [ 9362.835037] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [ 9378.870412] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 9380.201120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9380.629412] alg: No test for adler32 (adler32-zlib) [ 9381.755680] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 9381.931093] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [ 9383.607636] Key type lgssc registered [ 9385.041910] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9414.806388] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [ 9433.867422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9433.888671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9435.086801] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9435.108897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9435.175762] Lustre: lustre-MDT0000: new disk, initializing [ 9435.242276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9435.268590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9440.377729] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9451.821347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9452.087862] Lustre: lustre-OST0000: new disk, initializing [ 9452.093829] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9452.100248] Lustre: Skipped 1 previous similar message [ 9452.117901] Lustre: 298980:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9452.203395] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9459.202444] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [ 9461.389818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9461.410637] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9461.466105] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10279.260595] Lustre: server umount lustre-MDT0000 complete [10284.094296] LustreError: 298184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781239976 with bad export cookie 5870645845547747312 [10284.130716] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10294.967662] Lustre: server umount lustre-OST0000 complete [10323.436131] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [10333.513657] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [10374.312492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10374.694844] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10374.757954] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10374.927717] Lustre: lustre-MDT0000: new disk, initializing [10375.021356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10375.064656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10380.117554] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10392.066573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10392.160448] Lustre: 303266:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10392.202395] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10392.205746] Lustre: Skipped 1 previous similar message [10392.269148] Lustre: lustre-MDT0001: new disk, initializing [10392.349952] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10392.384044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10392.394437] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10397.012600] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10405.039200] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10409.273333] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10416.552679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10416.904372] Lustre: lustre-OST0000: new disk, initializing [10416.910265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10416.917468] Lustre: 305005:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10417.030043] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10423.144260] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10425.436355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10425.451170] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10425.580623] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10431.432812] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10436.641326] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10436.858825] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10441.921799] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10442.172750] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10446.313261] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10446.316729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10446.331515] LustreError: Skipped 1 previous similar message [10446.339223] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10446.371223] Lustre: Skipped 1 previous similar message [10446.399337] Lustre: Skipped 1 previous similar message [10450.451378] Lustre: server umount lustre-OST0000 complete [10456.545182] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10456.570672] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10456.575982] Lustre: Skipped 1 previous similar message [10456.599419] Lustre: Skipped 1 previous similar message [10461.670343] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10461.692112] Lustre: Skipped 1 previous similar message [10466.786087] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10466.796916] Lustre: Skipped 1 previous similar message [10469.347945] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10469.576938] Lustre: server umount lustre-MDT0000 complete [10474.258520] LustreError: 303258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240166 with bad export cookie 5870645845550688131 [10474.284862] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10474.675336] Lustre: server umount lustre-MDT0001 complete [10478.476874] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10480.340640] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:56:11 (1781240171) [10487.526204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10488.246677] LustreError: 307173:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10488.276814] LustreError: 307173:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10488.335175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10493.424884] LustreError: 307174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10493.720176] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10498.539317] LustreError: 307173:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10503.665288] LustreError: 307174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.387976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10505.945157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10509.012517] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10514.704684] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10519.504531] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10526.896705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10527.543564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10534.453354] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10540.314424] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10547.612782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10547.800129] Lustre: lustre-OST0001: new disk, initializing [10547.804456] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10547.809928] Lustre: 309667:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10547.955586] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10553.487244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10553.509249] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10553.594207] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10553.611761] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10554.663796] LustreError: 309915:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10554.678764] LustreError: 309915:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10555.547127] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10555.689982] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10557.744158] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10557.755810] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10561.908837] LustreError: 309915:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10561.915711] LustreError: 309915:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10565.053870] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10565.258273] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10569.214805] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c74c2b9c800.ost_server_uuid 50 [10570.150573] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10570.182335] LustreError: 309930:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10571.119777] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c74c2b9c800.ost_server_uuid in FULL state after 0 sec [10576.055803] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c74c2b9c800.ost_server_uuid 50 [10577.907874] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c74c2b9c800.ost_server_uuid in FULL state after 0 sec [10579.501286] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10594.793042] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10594.796738] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10594.811869] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10594.829221] Lustre: Skipped 1 previous similar message [10594.845198] Lustre: Skipped 1 previous similar message [10605.030275] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10605.040127] Lustre: Skipped 5 previous similar messages [10607.071130] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10607.186887] Lustre: server umount lustre-OST0001 complete [10610.145711] LustreError: 309471:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10610.158420] LustreError: 309471:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10612.706386] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10612.719232] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10618.744807] Lustre: server umount lustre-OST0000 complete [10623.472478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10623.481756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10623.501231] Lustre: Skipped 1 previous similar message [10623.504082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10623.509347] Lustre: Skipped 4 previous similar messages [10628.900876] Lustre: server umount lustre-MDT0000 complete [10630.628995] LustreError: 307173:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10630.648826] LustreError: 307173:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10632.871483] LustreError: 309666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240324 with bad export cookie 5870645845550689013 [10632.879165] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10632.882612] LustreError: 309666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [10633.207441] Lustre: server umount lustre-MDT0001 complete [10643.762984] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [10646.078231] Key type lgssc unregistered [10646.251173] LNet: 312115:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10646.257957] LNetError: 312115:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10647.274771] LNet: Removed LNI 192.168.202.112@tcp [10648.228671] Key type .llcrypt unregistered [10648.230429] Key type ._llcrypt unregistered [10671.133658] Key type ._llcrypt registered [10671.137313] Key type .llcrypt registered [10671.226481] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [10672.365232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10672.549075] alg: No test for adler32 (adler32-zlib) [10673.753267] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [10674.028207] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [10675.687188] Key type lgssc registered [10677.041605] Lustre: Echo OBD driver; http://www.lustre.org/ [10686.252052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10686.267721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10687.938196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10693.186535] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10697.187918] LustreError: 313521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10697.213314] LustreError: 313521:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10699.287861] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10702.305750] LustreError: 313522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10706.165745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10706.591371] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10710.381605] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10719.458979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10719.745766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10724.700540] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10729.372909] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10731.046917] LustreError: 315070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10733.092819] LustreError: 315685:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10733.119099] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10733.121110] LustreError: 315685:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10735.333106] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10735.548793] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10740.743490] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10740.938871] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10747.003898] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10754.448617] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10754.696262] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10853.354107] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10853.355733] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10853.366190] LustreError: Skipped 1 previous similar message [10853.378842] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10853.391466] Lustre: Skipped 1 previous similar message [10853.412579] Lustre: Skipped 1 previous similar message [10857.783884] Lustre: server umount lustre-OST0001 complete [10858.477812] LustreError: 315072:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10858.507058] LustreError: 315072:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10866.147452] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10866.161237] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10866.172103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10868.733165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10868.733496] LustreError: 315685:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10868.736319] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10868.742381] Lustre: Skipped 1 previous similar message [10868.783171] LustreError: 315685:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10870.994209] Lustre: server umount lustre-OST0000 complete [10875.360296] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10875.374631] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10875.395757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10881.360795] Lustre: server umount lustre-MDT0000 complete [10884.805962] LustreError: 316844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240576 with bad export cookie 7256211607672131949 [10884.818441] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10885.273052] Lustre: server umount lustre-MDT0001 complete [10895.082943] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [10897.505767] Key type lgssc unregistered [10897.767242] LNet: 318086:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10897.773699] LNetError: 318086:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10898.790488] LNet: Removed LNI 192.168.202.112@tcp [10899.411373] Key type .llcrypt unregistered [10899.413493] Key type ._llcrypt unregistered [10943.047446] Key type ._llcrypt registered [10943.051826] Key type .llcrypt registered [10943.161813] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [10943.904551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10944.004752] alg: No test for adler32 (adler32-zlib) [10945.001632] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [10945.185948] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [10946.863211] Key type lgssc registered [10947.951619] Lustre: Echo OBD driver; http://www.lustre.org/ [10956.566974] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10956.589297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10958.276716] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10963.734729] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10968.552928] LustreError: 320362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10968.579053] LustreError: 320362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10973.676973] LustreError: 320361:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10973.679294] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10974.081100] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10978.730225] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10981.500502] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10988.499476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10988.881279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10995.253576] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [10999.163047] LustreError: 321817:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11003.283742] LustreError: 321817:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11003.297412] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11003.315036] LustreError: 321817:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11005.647667] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11006.082764] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11011.575734] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11012.491486] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11022.237084] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11027.897960] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11040.741055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11040.753613] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11040.781066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11042.284485] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11042.285357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11042.295291] Lustre: Skipped 1 previous similar message [11042.302429] Lustre: Skipped 1 previous similar message [11046.177784] Lustre: server umount lustre-MDT0000 complete [11047.398031] LustreError: 320362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11047.414423] LustreError: 320362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11049.899908] LustreError: 320342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240741 with bad export cookie 2306912603270409895 [11049.903706] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11049.912399] LustreError: 320342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11050.258983] Lustre: server umount lustre-MDT0001 complete [11064.511473] Lustre: server umount lustre-OST0000 complete [11078.493964] Lustre: server umount lustre-OST0001 complete [11098.141582] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [11108.972897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11109.301741] LustreError: 325540:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11109.387644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11114.770598] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11123.157756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11123.714564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11128.281280] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11131.344648] Lustre: 326647:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11137.894687] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11144.658992] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11149.611769] LustreError: 327003:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11149.661787] LustreError: 327003:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11153.677297] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11154.128646] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11154.133462] Lustre: Skipped 1 previous similar message [11155.211699] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11155.227827] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11160.219472] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11168.491311] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11172.205558] Lustre: 328486:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11180.000808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11180.019731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11180.069059] Lustre: Skipped 1 previous similar message [11180.087556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11180.091974] Lustre: Skipped 2 previous similar messages [11182.692274] Lustre: server umount lustre-MDT0000 complete [11186.148491] LustreError: 325540:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11186.198113] LustreError: 325540:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11187.801957] LustreError: 326649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240879 with bad export cookie 2306912603270411596 [11187.814394] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11187.818223] LustreError: 326649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11188.464098] Lustre: server umount lustre-MDT0001 complete [11203.793312] Lustre: server umount lustre-OST0000 complete [11207.647841] Lustre: 319625:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781240883/real 1781240883] req@ffff8a2b0d348380 x1867766182296576/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781240899 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11207.674623] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11207.682102] Lustre: Skipped 3 previous similar messages [11208.650398] Lustre: server umount lustre-OST0001 complete [11213.178406] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11214.822212] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11216.570700] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 01:08:27 (1781240907) [11241.590471] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [11251.876858] loop0: detected capacity change from 0 to 1099511627776 [11252.508503] SGI XFS with ACLs, security attributes, quota, no debug enabled [11252.527334] *** VALIDATE xfs *** [11252.566053] XFS (loop0): Mounting V5 Filesystem [11252.618588] XFS (loop0): Ending clean mount [11254.151682] loop1: detected capacity change from 0 to 18691697672192 [11360.176888] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [11368.419117] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [11406.756492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11407.065132] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11407.116209] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11407.255958] Lustre: lustre-MDT0000: new disk, initializing [11407.402309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11407.430561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11412.198037] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11422.954469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11423.024070] Lustre: 333572:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11423.061861] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11423.065362] Lustre: Skipped 1 previous similar message [11423.118873] Lustre: lustre-MDT0001: new disk, initializing [11423.192121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11423.204134] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11427.800331] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11435.952491] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11439.891822] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11446.186763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11446.528819] Lustre: lustre-OST0000: new disk, initializing [11446.539464] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11446.543368] Lustre: 335312:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11447.732632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11447.747708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11447.885443] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11453.198418] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11460.686115] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11465.323930] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11465.555903] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11471.459555] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11471.815279] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11478.496682] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11478.511761] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11478.534950] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11478.546825] Lustre: Skipped 5 previous similar messages [11481.110706] Lustre: server umount lustre-OST0000 complete [11488.746369] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11488.763930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11488.774259] Lustre: Skipped 2 previous similar messages [11488.788802] Lustre: Skipped 1 previous similar message [11492.825161] Lustre: server umount lustre-MDT0000 complete [11493.874111] LustreError: 333578:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11493.903291] LustreError: 333578:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11496.634393] LustreError: 335323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781241188 with bad export cookie 2306912603270413136 [11496.650413] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11496.940370] Lustre: server umount lustre-MDT0001 complete [11505.017663] XFS (loop0): Unmounting Filesystem [11510.164692] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 01:13:21 (1781241201) [11517.703707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11518.081669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11518.085623] Lustre: Skipped 2 previous similar messages [11522.702084] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11531.861892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11536.502554] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11541.849164] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11545.673574] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11553.439515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11559.955870] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11564.922505] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11573.222090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11573.222520] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11573.245230] LustreError: Skipped 2 previous similar messages [11573.246927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11573.275906] Lustre: Skipped 1 previous similar message [11579.233874] Lustre: server umount lustre-OST0000 complete [11583.456371] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11588.857493] Lustre: server umount lustre-MDT0000 complete [11592.354968] LustreError: 340263:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781241284 with bad export cookie 2306912603270414018 [11592.355862] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11592.367982] LustreError: 340263:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [11592.753429] Lustre: server umount lustre-MDT0001 complete [11602.841157] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [11605.432964] Key type lgssc unregistered [11605.656892] LNet: 340997:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11605.662300] LNetError: 340997:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11605.678246] LNet: Removed LNI 192.168.202.112@tcp [11606.417267] Key type .llcrypt unregistered [11606.418993] Key type ._llcrypt unregistered [11636.334656] Key type ._llcrypt registered [11636.336739] Key type .llcrypt registered [11636.416644] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [11637.254111] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11637.313416] alg: No test for adler32 (adler32-zlib) [11638.350795] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [11638.544491] LNet: Added LNI 192.168.202.112@tcp [8/256/0/180] [11640.175189] Key type lgssc registered [11641.316558] Lustre: Echo OBD driver; http://www.lustre.org/ [11648.298521] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11648.318709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11649.706974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11654.164482] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11659.236263] LustreError: 342550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11659.247530] LustreError: 342550:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11661.715336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11661.974613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11666.175461] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11671.960412] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11676.333199] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11682.878892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11683.223786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11689.362444] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11694.367387] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11700.920545] Lustre: server umount lustre-MDT0001 complete [11702.096382] LustreError: 342546:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.12@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11703.264510] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11703.280963] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11703.307796] LustreError: 342550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11704.803548] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11707.203036] LustreError: 342544:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.12@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11707.216812] LustreError: 342544:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11709.943938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11710.078307] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11710.082313] Lustre: Found index 1 for lustre-MDT0001, updating log [11710.134448] Lustre: 342537:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11710.460604] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11714.358395] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [11715.573521] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11717.147561] LustreError: 345792:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11717.162086] LustreError: 345792:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11717.170254] LustreError: 342541:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11717.250186] LustreError: 345795:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -17 [11717.258189] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11717.258189] [12019.569730] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [12022.765460] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12022.778142] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12022.780773] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12022.785322] Lustre: Skipped 1 previous similar message [12022.806831] Lustre: Skipped 1 previous similar message [12024.641970] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [12025.103254] Lustre: server umount lustre-MDT0001 complete [12027.873449] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12027.915405] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12036.573617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12036.742104] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12036.750502] Lustre: Found index 1 for lustre-MDT0001, updating log [12036.860124] Lustre: 342537:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12037.218529] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12041.762682] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [12042.255257] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12043.769231] LustreError: 354150:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12043.782244] LustreError: 354150:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12043.788556] LustreError: 342541:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12043.793026] LustreError: 342541:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12043.876754] LustreError: 354159:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12043.884816] LustreError: 354159:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12043.888887] LustreError: 354159:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -17 [12043.903114] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.112@tcp [12043.903114] [12347.230160] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [12349.408411] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12349.414858] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12349.448138] Lustre: Skipped 1 previous similar message [12352.319607] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.12@tcp (stopping) [12352.330151] Lustre: Skipped 2 previous similar messages [12353.333637] Lustre: server umount lustre-MDT0001 complete [12354.531197] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12354.547318] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [12364.228701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12364.346284] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12364.352742] Lustre: Found index 1 for lustre-MDT0001, updating log [12364.410591] Lustre: 342539:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12364.792156] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12369.334459] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [12369.917305] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12373.496812] LustreError: 363008:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12373.507537] LustreError: 363008:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12373.515239] LustreError: 342541:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12373.518821] LustreError: 342541:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12373.566908] LustreError: 363011:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12373.578047] LustreError: 363011:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.112@tcp: cfg command failed: rc = -114 [12373.583743] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12373.583743] [12692.449308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12692.460986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12692.482291] Lustre: Skipped 2 previous similar messages [12692.499535] Lustre: Skipped 1 previous similar message [12696.900847] Lustre: server umount lustre-MDT0000 complete [12697.577743] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12697.601477] LustreError: 342549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [12700.944657] LustreError: 342530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781242393 with bad export cookie 6126717187574001694 [12700.951514] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12700.960970] LustreError: 342530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12702.691600] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12707.568909] Lustre: server umount lustre-MDT0001 complete [12711.696405] Lustre: server umount lustre-OST0000 complete [12721.911389] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [12731.633645] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [12771.445475] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:34:22 (1781242462) [12777.708155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12777.978672] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12778.017574] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12778.138788] Lustre: lustre-MDT0000: new disk, initializing [12778.208744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12778.230212] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12782.520613] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [12793.357369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12793.574599] Lustre: 373554:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12793.649427] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12793.653535] Lustre: Skipped 1 previous similar message [12793.745171] Lustre: lustre-MDT0001: new disk, initializing [12793.882477] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12793.937890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12793.946192] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12798.719379] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [12806.638944] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12810.782276] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12818.602709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12818.843861] Lustre: lustre-OST0000: new disk, initializing [12818.855618] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12818.863703] Lustre: 375294:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12818.987122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12825.637193] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_default_debug -1 all [12827.165219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12827.172515] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12827.354613] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12833.396905] Lustre: DEBUG MARKER: oleg212-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12841.956630] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12841.966132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12841.984163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12841.995122] Lustre: Skipped 2 previous similar messages [12842.734819] Lustre: server umount lustre-MDT0000 complete [12843.003360] LustreError: 373565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12843.019480] LustreError: 373565:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12846.745400] LustreError: 373546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781242538 with bad export cookie 6126717187574003640 [12846.752941] LustreError: 374343:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12846.775413] LustreError: MGC192.168.202.112@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12847.059660] Lustre: server umount lustre-MDT0001 complete [12861.181578] Lustre: server umount lustre-OST0000 complete [12894.647214] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing set_hostid [12903.928257] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing load_modules_local [12932.779965] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12599 sec ========= 01:37:04 (1781242624) [12934.411934] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:37:05 (1781242625) === [12937.653737] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:37:08 (1781242628) === [12974.096781] Lustre: DEBUG MARKER: oleg212-server.virtnet: executing unload_modules_local [12976.695009] Key type lgssc unregistered [12977.091705] LNet: 381497:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12977.108111] LNetError: 381497:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12977.119613] LNet: Removed LNI 192.168.202.112@tcp [12978.015671] Key type .llcrypt unregistered [12978.017444] Key type ._llcrypt unregistered