[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 524533319 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003319] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.007817] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010147] LSM: Security Framework initializing [ 0.011086] Yama: becoming mindful. [ 0.012049] SELinux: Initializing. [ 0.013109] *** VALIDATE selinux *** [ 0.022253] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027477] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028188] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029130] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030153] *** VALIDATE tmpfs *** [ 0.032008] *** VALIDATE proc *** [ 0.033220] *** VALIDATE cgroup *** [ 0.034015] *** VALIDATE cgroup2 *** [ 0.035318] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036181] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039035] Spectre V2 : User space: Vulnerable [ 0.040011] Speculative Store Bypass: Vulnerable [ 0.043525] debug: unmapping init [mem 0xffffffffb7e59000-0xffffffffb7e60fff] [ 0.045222] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046742] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047033] ... version: 2 [ 0.048020] ... bit width: 48 [ 0.049018] ... generic registers: 4 [ 0.050020] ... value mask: 0000ffffffffffff [ 0.051019] ... max period: 00007fffffffffff [ 0.052019] ... fixed-purpose events: 3 [ 0.053016] ... event mask: 000000070000000f [ 0.054310] rcu: Hierarchical SRCU implementation. [ 0.056708] smp: Bringing up secondary CPUs ... [ 0.057666] x86: Booting SMP configuration: [ 0.058036] .... node #0, CPUs: #1 #2 #3 [ 0.061425] smp: Brought up 1 node, 4 CPUs [ 0.063015] smpboot: Max logical packages: 1 [ 0.064020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.150249] node 0 deferred pages initialised in 83ms [ 0.155012] devtmpfs: initialized [ 0.157355] x86/mm: Memory block size: 128MB [ 0.160000] gcov: version magic: 0x41383552 [ 0.163330] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.167114] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.169318] pinctrl core: initialized pinctrl subsystem [ 0.171220] [ 0.171827] ************************************************************* [ 0.175014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.177017] ** ** [ 0.180017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.182012] ** ** [ 0.185017] ** This means that this kernel is built to expose internal ** [ 0.187013] ** IOMMU data structures, which may compromise security on ** [ 0.190017] ** your system. ** [ 0.192012] ** ** [ 0.195015] ** If you see this message and you are not debugging the ** [ 0.197011] ** kernel, report this immediately to your vendor! ** [ 0.199016] ** ** [ 0.202013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.204013] ************************************************************* [ 0.206709] NET: Registered protocol family 16 [ 0.209446] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.212069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.214065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.218116] cpuidle: using governor menu [ 0.220289] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.223361] PCI: Using configuration type 1 for base access [ 0.226137] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.238062] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.241117] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.246192] cryptd: max_cpu_qlen set to 1000 [ 0.250264] ACPI: Added _OSI(Module Device) [ 0.252015] ACPI: Added _OSI(Processor Device) [ 0.255019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.257038] ACPI: Added _OSI(Processor Aggregator Device) [ 0.262363] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.269542] ACPI: Interpreter enabled [ 0.271065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.272014] ACPI: Using IOAPIC for interrupt routing [ 0.274124] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.278394] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.288630] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.291041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.294027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.298118] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.303346] acpiphp: Slot [2] registered [ 0.305127] acpiphp: Slot [5] registered [ 0.306116] acpiphp: Slot [6] registered [ 0.307136] acpiphp: Slot [7] registered [ 0.309114] acpiphp: Slot [8] registered [ 0.310116] acpiphp: Slot [9] registered [ 0.311094] acpiphp: Slot [10] registered [ 0.312111] acpiphp: Slot [3] registered [ 0.314109] acpiphp: Slot [4] registered [ 0.315143] acpiphp: Slot [11] registered [ 0.317117] acpiphp: Slot [12] registered [ 0.319133] acpiphp: Slot [13] registered [ 0.320120] acpiphp: Slot [14] registered [ 0.322121] acpiphp: Slot [15] registered [ 0.323132] acpiphp: Slot [16] registered [ 0.325098] acpiphp: Slot [17] registered [ 0.326117] acpiphp: Slot [18] registered [ 0.328116] acpiphp: Slot [19] registered [ 0.329085] acpiphp: Slot [20] registered [ 0.331137] acpiphp: Slot [21] registered [ 0.333123] acpiphp: Slot [22] registered [ 0.335119] acpiphp: Slot [23] registered [ 0.337124] acpiphp: Slot [24] registered [ 0.339117] acpiphp: Slot [25] registered [ 0.341112] acpiphp: Slot [26] registered [ 0.342123] acpiphp: Slot [27] registered [ 0.344121] acpiphp: Slot [28] registered [ 0.346107] acpiphp: Slot [29] registered [ 0.348144] acpiphp: Slot [30] registered [ 0.349090] acpiphp: Slot [31] registered [ 0.351087] PCI host bridge to bus 0000:00 [ 0.353025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.356038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.358021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.360018] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.362026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.365031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.367182] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.370028] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.374345] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.384014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.388950] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.391022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.393017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.396016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.398539] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.401827] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.405047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.408986] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.414012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.426016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.432021] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.436942] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.447017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.461020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.486023] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.497863] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.511019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.527026] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.563018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.575154] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.582015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.588018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.608016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.620378] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.627016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.640018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.660022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.676205] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.688018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.701021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.721021] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.730098] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.735012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.742023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.765022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.787853] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.791383] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.793835] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.797397] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.799241] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.804085] iommu: Default domain type: Passthrough [ 0.806428] SCSI subsystem initialized [ 0.808166] ACPI: bus type USB registered [ 0.810123] usbcore: registered new interface driver usbfs [ 0.812094] usbcore: registered new interface driver hub [ 0.814102] usbcore: registered new device driver usb [ 0.816186] pps_core: LinuxPPS API ver. 1 registered [ 0.818013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.822063] PTP clock support registered [ 0.824077] EDAC MC: Ver: 3.0.0 [ 0.826018] PCI: Using ACPI for IRQ routing [ 0.828028] NetLabel: Initializing [ 0.829012] NetLabel: domain hash size = 128 [ 0.831013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.833080] NetLabel: unlabeled traffic allowed by default [ 0.836017] vgaarb: loaded [ 0.838303] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.840012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.848959] clocksource: Switched to clocksource kvm-clock [ 0.959275] VFS: Disk quotas dquot_6.6.0 [ 0.961562] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.964432] *** VALIDATE ramfs *** [ 0.965738] *** VALIDATE hugetlbfs *** [ 0.968085] pnp: PnP ACPI init [ 0.970553] pnp: PnP ACPI: found 6 devices [ 1.028182] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.032604] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.035174] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.037879] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.040661] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.043251] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.046203] NET: Registered protocol family 2 [ 1.048644] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.053737] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.057597] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.063316] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.067669] TCP: Hash tables configured (established 65536 bind 65536) [ 1.071327] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.075219] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.078694] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.082409] NET: Registered protocol family 1 [ 1.086109] RPC: Registered named UNIX socket transport module. [ 1.088631] RPC: Registered udp transport module. [ 1.090624] RPC: Registered tcp transport module. [ 1.092769] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.095841] NET: Registered protocol family 44 [ 1.097575] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.099931] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.102080] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.104854] PCI: CLS 0 bytes, default 64 [ 1.107278] Unpacking initramfs... [ 2.534217] debug: unmapping init [mem 0xffff904ebcc54000-0xffff904ebffbffff] [ 2.538177] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.540540] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.543485] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.056698] Initialise system trusted keyrings [ 3.058600] Key type blacklist registered [ 3.060472] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.071365] zbud: loaded [ 3.074653] *** VALIDATE nfs *** [ 3.076133] *** VALIDATE nfs4 *** [ 3.077936] pstore: using deflate compression [ 3.081454] Platform Keyring initialized [ 3.178935] NET: Registered protocol family 38 [ 3.180635] Key type asymmetric registered [ 3.182052] Asymmetric key parser 'x509' registered [ 3.184451] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.187750] io scheduler mq-deadline registered [ 3.189300] io scheduler kyber registered [ 3.191540] io scheduler bfq registered [ 3.193715] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.196875] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.199886] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.202077] ACPI: Power Button [PWRF] [ 3.207511] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.214344] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.227906] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.235080] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.248831] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.277612] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.306446] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.313170] Non-volatile memory driver v1.3 [ 3.314848] Linux agpgart interface v0.103 [ 3.345713] virtio_blk virtio1: [vda] 150608 512-byte logical blocks (77.1 MB/73.5 MiB) [ 3.348617] vda: detected capacity change from 0 to 77111296 [ 3.368359] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.371556] vdb: detected capacity change from 0 to 1073741824 [ 3.389482] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.392011] vdc: detected capacity change from 0 to 2621440000 [ 3.405896] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.407898] vdd: detected capacity change from 0 to 2621440000 [ 3.418517] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.420568] vde: detected capacity change from 0 to 4294967296 [ 3.432159] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.434028] vdf: detected capacity change from 0 to 4294967296 [ 3.441147] libphy: Fixed MDIO Bus: probed [ 3.446156] usbcore: registered new interface driver usbserial_generic [ 3.448953] usbserial: USB Serial support registered for generic [ 3.451223] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.455837] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.458017] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.460817] mousedev: PS/2 mouse device common for all mice [ 3.463033] rtc_cmos 00:05: RTC can wake from S4 [ 3.465410] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.467931] rtc_cmos 00:05: registered as rtc0 [ 3.471126] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.474558] intel_pstate: CPU model not supported [ 3.477922] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.481311] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.482478] hid: raw HID events driver (C) Jiri Kosina [ 3.486303] usbcore: registered new interface driver usbhid [ 3.489629] usbhid: USB HID core driver [ 3.492211] drop_monitor: Initializing network drop monitor service [ 3.495789] Initializing XFRM netlink socket [ 3.498664] NET: Registered protocol family 10 [ 3.502327] Segment Routing with IPv6 [ 3.503217] NET: Registered protocol family 17 [ 3.504456] mpls_gso: MPLS GSO support [ 3.509724] RAS: Correctable Errors collector initialized. [ 3.511062] AVX version of gcm_enc/dec engaged. [ 3.512088] AES CTR mode by8 optimization enabled [ 3.575864] sched_clock: Marking stable (3575838110, 0)->(4497807260, -921969150) [ 3.578561] registered taskstats version 1 [ 3.579911] Loading compiled-in X.509 certificates [ 3.581232] zswap: loaded using pool lzo/zbud [ 3.605334] Key type big_key registered [ 3.616455] Key type encrypted registered [ 3.617638] ima: No TPM chip found, activating TPM-bypass! [ 3.619482] ima: Allocated hash algorithm: sha1 [ 3.621249] ima: No architecture policies found [ 3.622330] evm: Initialising EVM extended attributes: [ 3.623668] evm: security.selinux [ 3.624394] evm: security.ima [ 3.625449] evm: security.capability [ 3.626994] evm: HMAC attrs: 0x1 [ 3.629547] rtc_cmos 00:05: setting system clock to 2026-10-02 22:14:40 UTC (1790979280) [ 3.635969] debug: unmapping init [mem 0xffffffffb8e03000-0xffffffffb8ffffff] [ 3.639162] debug: unmapping init [mem 0xffffffffb7b82000-0xffffffffb7e58fff] [ 3.650534] Write protecting the kernel read-only data: 28672k [ 3.653301] debug: unmapping init [mem 0xffffffffb6203000-0xffffffffb63fffff] [ 3.656108] debug: unmapping init [mem 0xffffffffb6b14000-0xffffffffb6bfffff] [ 3.691703] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.700184] systemd[1]: Detected virtualization kvm. [ 3.702478] systemd[1]: Detected architecture x86-64. [ 3.704299] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.727389] systemd[1]: No hostname configured. [ 3.728670] systemd[1]: Set hostname to . [ 3.729961] random: systemd: uninitialized urandom read (16 bytes read) [ 3.732108] systemd[1]: Initializing machine ID from random generator. [ 3.776719] random: ln: uninitialized urandom read (6 bytes read) [ 3.845269] random: systemd: uninitialized urandom read (16 bytes read) [ 3.846886] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.849694] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.851746] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Reached target Swap. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.391779] device-mapper: uevent: version 1.0.3 [ 4.394658] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.132879] random: fast init done [ 5.152747] virtio_net virtio0 ens2: renamed from eth0 [ 5.183390] scsi host0: ata_piix [ 5.250085] scsi host1: ata_piix [ 5.253251] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.255949] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.919055] dracut-initqueue[596]: RTNETLINK answers: File exists [ 10.131444] random: crng init done [ 10.132970] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.443315] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.516178] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.771629] SELinux: Disabled at runtime. [ 11.832371] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.844748] systemd[1]: Detected virtualization kvm. [ 11.847077] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.236531] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.238886] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.243392] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.247339] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.250832] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.257876] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.264826] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Local Encrypted Volumes. Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting udev Coldplug all Devices... [ 12.420296] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.700821] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.994105] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.078262] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.197878] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.212771] EDAC sbridge: Ver: 1.1.2 [ 14.882445] Key type dns_resolver registered [ 15.182266] NFS: Registering the id_resolver key type [ 15.183975] Key type id_resolver registered [ 15.185079] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg255-server login: [ 41.422221] libcfs: loading out-of-tree module taints kernel. [ 41.441756] Key type ._llcrypt registered [ 41.443283] Key type .llcrypt registered [ 41.490290] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 61.472970] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 63.580193] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 63.596777] alg: No test for adler32 (adler32-zlib) [ 65.154629] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 66.423945] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 68.312613] Key type lgssc registered [ 70.572188] Lustre: Echo OBD driver; http://www.lustre.org/ [ 90.560485] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 106.006249] hrtimer: interrupt took 3547553 ns [ 136.331387] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 150.412827] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 150.468369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 151.735434] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 151.760166] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 151.847453] Lustre: lustre-MDT0000: new disk, initializing [ 151.927740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 151.950101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 158.082387] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 174.525066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 174.659831] Lustre: 6520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 174.690616] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 174.697271] Lustre: Skipped 1 previous similar message [ 174.790609] Lustre: lustre-MDT0001: new disk, initializing [ 174.909788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 174.950915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 174.974082] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 180.878369] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 185.883696] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 200.853980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 201.343407] Lustre: lustre-OST0000: new disk, initializing [ 201.354989] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 201.374526] Lustre: 8462:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 201.455956] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 209.725545] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 210.505727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 210.513361] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 210.624711] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 224.950163] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.176305] Lustre: lustre-OST0001: new disk, initializing [ 225.185159] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.198774] Lustre: 9538:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.361457] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 231.958284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 231.970290] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 232.047798] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 233.454612] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 248.647744] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 255.327611] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 262.358243] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing check_logdir /tmp/testlogs/ [ 268.395920] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing yml_node [ 274.100242] Lustre: DEBUG MARKER: Client: 2.17.59.2 [ 276.678822] Lustre: DEBUG MARKER: MDS: 2.17.59.2 [ 279.046097] Lustre: DEBUG MARKER: OSS: 2.17.59.2 [ 280.699458] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Oct 2 18:19:15 EDT 2026 [ 301.408970] Lustre: DEBUG MARKER: only running tests: 76-120 [ 303.240118] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 305.536067] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 323.552156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 323.558869] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 323.575590] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.068074] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.085441] Lustre: Skipped 1 previous similar message [ 324.087204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.099759] Lustre: Skipped 3 previous similar messages [ 329.184557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 329.196898] Lustre: Skipped 3 previous similar messages [ 334.309576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 334.320738] Lustre: Skipped 3 previous similar messages [ 336.357358] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 336.634080] Lustre: server umount lustre-MDT0000 complete [ 340.670449] LustreError: 6513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790979617 with bad export cookie 5767257631152214779 [ 340.698839] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 344.543781] LustreError: 6528:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 344.544262] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 344.550027] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 344.567293] LustreError: 6528:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 344.602981] Lustre: Skipped 2 previous similar messages [ 345.568091] LustreError: 6529:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 345.590436] LustreError: 6529:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 350.688370] LustreError: 6528:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 350.713296] LustreError: 6528:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 355.295183] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 355.560046] Lustre: server umount lustre-MDT0001 complete [ 360.131538] Lustre: server umount lustre-OST0000 complete [ 364.239502] Lustre: server umount lustre-OST0001 complete [ 371.436245] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 381.384211] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 419.566505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 419.838431] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 419.871508] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 419.946449] Lustre: lustre-MDT0000: new disk, initializing [ 420.004276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 420.029924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 425.123415] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 436.270231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 436.454456] Lustre: 15140:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 436.555451] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 436.564697] Lustre: Skipped 1 previous similar message [ 436.710466] Lustre: lustre-MDT0001: new disk, initializing [ 436.988914] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 437.081971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 437.098382] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 442.349422] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 454.706493] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 463.288553] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 471.140355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 471.444456] Lustre: lustre-OST0000: new disk, initializing [ 471.448840] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 471.457054] Lustre: 17108:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 471.576048] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 473.354530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 473.383870] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 473.541340] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 478.907810] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 488.920766] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 495.181459] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 495.452383] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 500.944306] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 501.164451] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 504.290678] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 504.316788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 504.319299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 504.329630] Lustre: Skipped 5 previous similar messages [ 509.411592] LustreError: 17092:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.427411] LustreError: 17092:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 509.506488] Lustre: server umount lustre-OST0000 complete [ 514.016513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 514.030066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 514.047215] Lustre: Skipped 1 previous similar message [ 519.631416] Lustre: server umount lustre-MDT0000 complete [ 519.654320] LustreError: 15152:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.853012] LustreError: 15133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790979800 with bad export cookie 5767257631152216312 [ 523.868528] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 523.875213] LustreError: 15133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 524.357545] Lustre: server umount lustre-MDT0001 complete [ 530.926618] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:23:25 (1790979805) [ 538.297835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 538.894440] LustreError: 19377:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 538.907650] LustreError: 19377:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 538.983555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 544.422524] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 553.234517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.446268] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 553.451193] Lustre: Skipped 2 previous similar messages [ 559.144160] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 567.912744] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 573.612565] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 581.395914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 586.973299] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 595.779765] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 603.243902] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 613.858144] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 613.866831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 613.878665] Lustre: Skipped 2 previous similar messages [ 617.310078] Lustre: server umount lustre-MDT0000 complete [ 618.991753] LustreError: 19373:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 619.018574] LustreError: 19373:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 620.930889] LustreError: 19358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790979897 with bad export cookie 5767257631152217194 [ 620.935359] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 620.953272] LustreError: 19358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 621.266907] Lustre: server umount lustre-MDT0001 complete [ 636.558512] Lustre: server umount lustre-OST0000 complete [ 659.197710] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 671.955499] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 672.398530] LustreError: 23760:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 672.484908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 672.494217] Lustre: Skipped 2 previous similar messages [ 677.300560] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 688.366306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 694.310945] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 697.124301] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 704.650742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 712.282623] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 721.839472] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 722.077746] Lustre: lustre-OST0001: new disk, initializing [ 722.081763] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 722.096651] Lustre: 26076:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 724.113750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 724.140731] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 724.300145] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 729.337109] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 739.158628] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 743.417367] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 755.171140] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 755.178852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 755.195894] Lustre: Skipped 3 previous similar messages [ 755.211673] Lustre: Skipped 5 previous similar messages [ 761.246660] Lustre: server umount lustre-MDT0000 complete [ 764.772093] LustreError: 23742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980041 with bad export cookie 5767257631152218034 [ 764.774983] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 764.798731] LustreError: 23742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 765.358567] Lustre: server umount lustre-MDT0001 complete [ 779.505863] Lustre: server umount lustre-OST0000 complete [ 781.600315] Lustre: 3651:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790980042/real 1790980042] req@ffff904f03851880 x1877977955815680/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790980058 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 783.380702] Lustre: server umount lustre-OST0001 complete [ 801.290386] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 812.885691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 813.291220] LustreError: 29094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 813.306869] LustreError: 29094:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 813.396637] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 813.402492] Lustre: Skipped 3 previous similar messages [ 819.082174] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 829.962830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 835.466724] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 838.749756] Lustre: 30245:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 847.087959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 856.583715] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 866.072075] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 874.154382] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 882.616489] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 886.611407] Lustre: 32125:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 897.002176] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 897.007626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 897.029504] Lustre: Skipped 3 previous similar messages [ 897.046986] Lustre: Skipped 7 previous similar messages [ 900.476584] Lustre: server umount lustre-MDT0000 complete [ 904.390755] LustreError: 29075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980181 with bad export cookie 5767257631152219336 [ 904.395433] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 904.400017] LustreError: 29075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 904.892214] Lustre: server umount lustre-MDT0001 complete [ 919.828041] Lustre: server umount lustre-OST0000 complete [ 933.725701] Lustre: server umount lustre-OST0001 complete [ 943.074440] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:30:17 (1790980217) [ 974.474940] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 986.880210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 987.744801] LustreError: 34931:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 987.770171] LustreError: 34931:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 12 previous similar messages [ 993.262483] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1003.165116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1009.140531] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1012.858611] Lustre: 36083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1020.070443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1027.898868] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1037.569192] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1044.031589] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1051.845901] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1055.604254] Lustre: 37960:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1063.394156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1063.416980] Lustre: Skipped 5 previous similar messages [ 1067.084776] Lustre: server umount lustre-MDT0000 complete [ 1071.868155] LustreError: 34911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980348 with bad export cookie 5767257631152220442 [ 1071.868479] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1071.881196] LustreError: 34911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1072.278427] Lustre: server umount lustre-MDT0001 complete [ 1085.317222] Lustre: server umount lustre-OST0000 complete [ 1098.424471] Lustre: server umount lustre-OST0001 complete [ 1109.046773] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:33:02 (1790980382) [ 1125.309637] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1136.348924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1136.983650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1136.994869] Lustre: Skipped 7 previous similar messages [ 1141.777359] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1152.458142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1157.585784] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1160.704336] Lustre: 41251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1168.110796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1174.496695] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1184.434373] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1192.145037] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1200.828803] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1204.532394] Lustre: 43128:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1205.705903] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1205.709070] Lustre: Skipped 1 previous similar message [ 1210.476960] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.55@tcp (stopping) [ 1210.488784] Lustre: Skipped 4 previous similar messages [ 1210.854875] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1210.890180] Lustre: Skipped 2 previous similar messages [ 1215.879685] Lustre: server umount lustre-MDT0000 complete [ 1220.530047] LustreError: 40083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980497 with bad export cookie 5767257631152221548 [ 1220.532253] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1220.558434] LustreError: 40083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1227.108898] Lustre: server umount lustre-MDT0001 complete [ 1236.460861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1245.664366] LustreError: 3650:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff904f32e49c00 x1877977955972992/t0(0) o250->MGC192.168.202.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1245.942732] LustreError: 41616:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1245.958882] LustreError: 41616:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 27 previous similar messages [ 1252.680608] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1255.282876] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1261.668601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1266.692741] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1267.714848] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1267.719804] Lustre: Skipped 1 previous similar message [ 1275.226500] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1277.030513] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1284.384157] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1286.601613] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1292.770175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1298.512541] Lustre: server umount lustre-MDT0000 complete [ 1302.757344] LustreError: 43130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980579 with bad export cookie 5767257631152222675 [ 1302.760837] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1302.762399] LustreError: 43130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1302.974581] Lustre: server umount lustre-MDT0001 complete [ 1317.179598] Lustre: server umount lustre-OST0000 complete [ 1319.712160] Lustre: 3653:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790980580/real 1790980580] req@ffff904f3260b100 x1877977956010880/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790980596 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1321.381668] Lustre: server umount lustre-OST0001 complete [ 1330.894515] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:36:45 (1790980605) [ 1348.555161] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1360.410452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1365.482913] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1374.354292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1379.682440] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1382.761769] Lustre: 48894:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1389.644111] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1397.045467] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1406.823778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1414.670088] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1421.918328] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1426.674639] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1441.252482] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1445.266399] Lustre: server umount lustre-MDT0000 complete [ 1448.626250] LustreError: 50089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790980725 with bad export cookie 5767257631152223466 [ 1448.632315] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1448.636787] LustreError: 50089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1448.919861] Lustre: server umount lustre-MDT0001 complete [ 1463.045986] Lustre: server umount lustre-OST0000 complete [ 1477.749674] Lustre: server umount lustre-OST0001 complete [ 1487.259394] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:39:21 (1790980761) [ 1490.261788] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1492.482675] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:39:26 (1790980766) [ 1502.849636] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1511.198647] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1534.762562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1534.997934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1535.076659] Lustre: lustre-MDT0000: new disk, initializing [ 1535.200311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1541.322813] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1552.166885] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1559.052401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1559.425500] Lustre: lustre-OST0000: new disk, initializing [ 1559.439906] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1559.451787] Lustre: Skipped 1 previous similar message [ 1559.472234] Lustre: 54971:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1561.215808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1561.231800] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1561.281088] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1566.544798] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1577.516648] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1580.107668] Lustre: DEBUG MARKER: create test files [ 1612.256809] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1612.272838] Lustre: Skipped 12 previous similar messages [ 1615.997074] Lustre: server umount lustre-OST0000 complete [ 1621.078302] Lustre: server umount lustre-MDT0000 complete [ 1633.247251] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1636.313155] Key type lgssc unregistered [ 1636.584499] LNet: 56600:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1636.588740] LNetError: 56600:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1637.606777] LNet: Removed LNI 192.168.202.155@tcp [ 1638.906167] Key type .llcrypt unregistered [ 1638.908945] Key type ._llcrypt unregistered [ 1671.298180] Key type ._llcrypt registered [ 1671.300937] Key type .llcrypt registered [ 1671.497196] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1672.805239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1672.847393] alg: No test for adler32 (adler32-zlib) [ 1673.984266] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 1674.140590] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1675.815975] Key type lgssc registered [ 1677.054671] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1686.443043] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1686.519613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1688.166814] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1693.599340] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1701.921761] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1709.867890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1710.597557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1712.626744] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1712.816724] Lustre: 59350:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1712.831920] Lustre: 59350:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1712.841456] Lustre: 59350:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1712.851488] Lustre: 59350:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1712.864184] Lustre: 59350:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1712.879273] Lustre: 59350:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1718.033872] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1726.966429] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1732.942138] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1738.688552] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1748.451749] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1748.452837] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1748.476505] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1753.570078] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1758.710354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1762.271163] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1762.513351] Lustre: server umount lustre-OST0000 complete [ 1767.154922] Lustre: server umount lustre-MDT0000 complete [ 1780.954990] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1784.018817] Key type lgssc unregistered [ 1784.292804] LNet: 60964:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1784.308078] LNetError: 60964:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1784.327324] LNet: Removed LNI 192.168.202.155@tcp [ 1785.370563] Key type .llcrypt unregistered [ 1785.376627] Key type ._llcrypt unregistered [ 1818.929230] Key type ._llcrypt registered [ 1818.937758] Key type .llcrypt registered [ 1819.060851] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1820.412097] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1820.543242] alg: No test for adler32 (adler32-zlib) [ 1821.767890] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 1822.073235] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1823.767294] Key type lgssc registered [ 1825.041596] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1833.260578] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1833.294203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1834.792988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1839.329163] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1848.032461] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1856.328052] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1856.726776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1859.842615] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1859.956823] Lustre: 63815:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1859.965283] Lustre: 63815:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1859.974227] Lustre: 63815:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1859.989100] Lustre: 63815:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1860.006214] Lustre: 63815:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1860.014548] Lustre: 63815:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1863.233763] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1871.274496] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1876.978860] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1885.666210] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1885.673590] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1885.691971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1890.785412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1891.659841] Lustre: server umount lustre-OST0000 complete [ 1895.652480] Lustre: server umount lustre-MDT0000 complete [ 1908.227651] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1911.005664] Key type lgssc unregistered [ 1911.293978] LNet: 65367:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1911.303468] LNetError: 65367:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1912.369618] LNet: Removed LNI 192.168.202.155@tcp [ 1913.327818] Key type .llcrypt unregistered [ 1913.330412] Key type ._llcrypt unregistered [ 1933.510577] Key type ._llcrypt registered [ 1933.513598] Key type .llcrypt registered [ 1933.673532] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1949.459302] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1950.311270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1950.320623] alg: No test for adler32 (adler32-zlib) [ 1951.475197] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 1951.831284] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1953.551930] Key type lgssc registered [ 1954.549572] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1997.265852] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1997.305812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1998.562896] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1998.588286] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1998.653364] Lustre: lustre-MDT0000: new disk, initializing [ 1998.732431] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1998.759794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2004.202978] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2015.406310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2015.495923] Lustre: 68541:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2015.538041] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2015.541784] Lustre: Skipped 1 previous similar message [ 2015.599468] Lustre: lustre-MDT0001: new disk, initializing [ 2015.712692] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2015.754208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2015.767846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2019.978603] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2030.130380] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2035.650210] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2043.509778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2043.855358] Lustre: lustre-OST0000: new disk, initializing [ 2043.860200] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2043.867677] Lustre: 70509:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2043.965050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2051.704701] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2052.140151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2052.149181] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2052.262875] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2061.261263] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2065.551043] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2065.794533] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2070.523373] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2070.831235] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2073.064714] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2073.073663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2073.084859] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2073.095748] Lustre: Skipped 1 previous similar message [ 2078.182176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2078.190506] Lustre: Skipped 1 previous similar message [ 2078.869344] Lustre: server umount lustre-OST0000 complete [ 2083.302134] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2083.321597] Lustre: Skipped 1 previous similar message [ 2083.324618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2088.419460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2088.437606] Lustre: Skipped 2 previous similar messages [ 2088.865608] Lustre: server umount lustre-MDT0000 complete [ 2092.873093] LustreError: 68530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790981369 with bad export cookie 5496495062768942279 [ 2092.884366] LustreError: 68530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2092.886477] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2093.382317] Lustre: server umount lustre-MDT0001 complete [ 2102.624883] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:49:36 (1790981376) [ 2114.868298] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2163.765588] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2174.719928] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2214.212180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2214.472515] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2214.491749] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2214.553995] Lustre: lustre-MDT0000: new disk, initializing [ 2214.608671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2214.625939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2219.450199] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2231.286924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2231.440378] Lustre: 76528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2231.532694] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2231.537163] Lustre: Skipped 1 previous similar message [ 2231.631316] Lustre: lustre-MDT0001: new disk, initializing [ 2231.826871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2231.910275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2231.937589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2237.415620] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2248.504377] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2254.316298] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2261.914430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2262.212841] Lustre: lustre-OST0000: new disk, initializing [ 2262.219134] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2262.232548] Lustre: 78498:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2262.379353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2268.923299] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2270.325252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2270.334435] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2270.477233] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2278.860295] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2284.803437] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2285.115912] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2290.462968] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2290.711291] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2296.288113] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2296.292647] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2296.304627] LustreError: Skipped 1 previous similar message [ 2296.312876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2296.331714] Lustre: Skipped 2 previous similar messages [ 2296.360237] Lustre: Skipped 1 previous similar message [ 2299.389690] Lustre: server umount lustre-OST0000 complete [ 2306.537848] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2306.550314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2306.559492] Lustre: Skipped 1 previous similar message [ 2309.935172] Lustre: server umount lustre-MDT0000 complete [ 2314.173807] LustreError: 76519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790981591 with bad export cookie 5496495062768943161 [ 2314.175861] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2314.187768] LustreError: 76519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2314.628875] Lustre: server umount lustre-MDT0001 complete [ 2324.044602] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:53:18 (1790981598) [ 2331.007271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2331.513667] LustreError: 80914:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2331.554846] LustreError: 80914:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2331.618829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2336.736137] LustreError: 80915:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2337.366482] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2341.857658] LustreError: 80914:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2346.985330] LustreError: 80915:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.083693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2352.177084] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2360.346216] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2367.351952] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2373.520692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2373.807021] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2373.817162] Lustre: Skipped 1 previous similar message [ 2379.417642] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2388.452944] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2392.954706] Lustre: 83585:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting f5aee61e-d45b-4b26-ac9c-1e04cecf7ded at adminstrative request [ 2397.669671] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2397.694240] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2407.910385] Lustre: MGS: Client f5aee61e-d45b-4b26-ac9c-1e04cecf7ded (at 0@lo) reconnecting [ 2407.915424] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4c4779234e0803ab to 0x4c4779234e08064b [ 2407.921119] Lustre: MGC192.168.202.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2431.753136] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2432.064096] Lustre: lustre-OST0001: new disk, initializing [ 2432.072465] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2432.078455] Lustre: 83979:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2439.438587] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2440.250831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2440.258459] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2440.374348] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2451.153177] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2456.034953] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2456.037711] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2456.040195] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2456.045147] Lustre: Skipped 2 previous similar messages [ 2469.343516] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2469.432270] Lustre: server umount lustre-OST0001 complete [ 2471.394955] LustreError: 82737:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2471.406545] LustreError: 82737:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2474.468080] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2474.477522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2481.638044] LustreError: 82734:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2481.645459] LustreError: 82734:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2487.263166] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2487.530774] Lustre: server umount lustre-OST0000 complete [ 2491.877330] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.889726] Lustre: Skipped 1 previous similar message [ 2491.892080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2491.907376] Lustre: Skipped 14 previous similar messages [ 2497.978885] Lustre: server umount lustre-MDT0000 complete [ 2500.066093] LustreError: 80915:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2500.104519] LustreError: 80915:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2502.589364] LustreError: 81650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790981779 with bad export cookie 5496495062768944715 [ 2502.591976] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2502.609904] LustreError: 81650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2503.319856] Lustre: server umount lustre-MDT0001 complete [ 2513.838071] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:56:27 (1790981787) [ 2515.904944] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2518.023218] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:56:32 (1790981792) [ 2519.724816] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2521.749660] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:56:35 (1790981795) [ 2523.573516] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2525.762741] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:56:39 (1790981799) [ 2527.960993] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2530.118410] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:56:44 (1790981804) [ 2531.895928] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2539.622769] loop: module loaded [ 2539.656064] loop0: detected capacity change from 0 to 4294967296 [ 2539.724300] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2541.645680] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2545.848502] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2550.018173] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2580.412227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2580.471947] LustreError: 88107:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2580.486908] LustreError: 88107:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2580.491361] LustreError: 88107:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2580.497318] LustreError: 88107:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2604.277588] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2616.985979] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2657.639553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2657.869955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2657.918692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2658.057041] Lustre: lustre-MDT0000: new disk, initializing [ 2658.193207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2658.202948] Lustre: Skipped 1 previous similar message [ 2658.229057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2663.650850] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2675.415087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2675.511794] Lustre: 90978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2675.763272] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2679.534178] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2689.915435] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2696.343599] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2704.718545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2705.144279] Lustre: 92946:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2712.140634] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2713.668451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2713.810541] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2721.734451] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2726.537615] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2726.858402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2732.601527] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2732.921342] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2739.681487] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2739.684911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2739.697068] Lustre: Skipped 1 previous similar message [ 2739.717703] Lustre: Skipped 3 previous similar messages [ 2741.168046] Lustre: server umount lustre-OST0000 complete [ 2751.063489] Lustre: server umount lustre-MDT0000 complete [ 2754.841094] LustreError: 92945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790982031 with bad export cookie 5496495062768945331 [ 2754.851574] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2754.865187] LustreError: 92945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2755.382818] Lustre: server umount lustre-MDT0001 complete [ 2764.615758] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:00:38 (1790982038) [ 2772.006997] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2772.467474] LustreError: 95358:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2777.545749] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2786.391159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2786.830837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2786.842619] Lustre: Skipped 3 previous similar messages [ 2791.346858] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2798.550062] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2804.024849] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2812.025048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2819.802467] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2827.502480] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2832.073955] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2832.296259] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2836.984571] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2837.172239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2843.480246] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2843.741892] Lustre: lustre-OST0001: new disk, initializing [ 2843.744233] Lustre: Skipped 2 previous similar messages [ 2843.748189] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2843.752617] Lustre: Skipped 3 previous similar messages [ 2843.757484] Lustre: 98741:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2849.219802] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2849.371528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2849.382663] Lustre: Skipped 2 previous similar messages [ 2849.391837] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2849.520526] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2858.689189] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2864.067031] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2864.397187] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2870.563948] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2870.752301] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2889.865418] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2914.306152] Lustre: Failing over lustre-MDT0000 [ 2914.724196] Lustre: server umount lustre-MDT0000 complete [ 2914.792410] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2914.798699] LustreError: Skipped 1 previous similar message [ 2914.804929] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2914.817916] Lustre: Skipped 2 previous similar messages [ 2914.824837] LustreError: 95358:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2914.842869] LustreError: 95358:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 2932.703371] Lustre: 66753:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790982193/real 1790982193] req@0000000062066ae8 x1877979932408576/t0(0) o400->MGC192.168.202.155@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790982209 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2932.736792] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2939.848140] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2939.850068] LDISKFS-fs (dm-0): recovery complete [ 2939.870217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2942.958803] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x4c4779234e08d0bf [ 2942.971259] Lustre: MGC192.168.202.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2943.388063] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2944.570548] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2944.572818] Lustre: 95354:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2948.319153] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2948.592956] Lustre: 96099:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2948.606781] Lustre: 96099:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2948.611136] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2948.651672] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2948.967415] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2949.302073] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2949.313835] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2949.631063] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2949.640154] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2949.654976] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2949.678854] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2950.336333] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2950.352173] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2950.679135] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2950.684913] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2951.961674] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2951.974835] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2952.615872] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2952.630308] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2952.951235] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2952.957581] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2956.168699] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2956.181479] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2956.824096] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2956.836587] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2957.151117] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2957.154049] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2964.177929] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2964.188658] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2964.838346] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2964.861181] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2965.191274] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2965.200697] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2980.196500] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2980.229066] Lustre: 101589:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 49 previous similar messages [ 2980.875589] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2980.886115] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 2981.207231] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2981.222033] LustreError: 101589:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3004.127625] Lustre: 101589:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3004.150473] Lustre: 101589:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3004.172619] Lustre: 101589:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 233d9212-169f-4c8e-a145-f186999f76de@192.168.202.55@tcp [ 3004.189917] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3004.204418] Lustre: 101589:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@000000001b7f891e x1877979915423232/t0(8589934770) o35->233d9212-169f-4c8e-a145-f186999f76de@192.168.202.55@tcp:736/0 lens 392/0 e 0 to 0 dl 1790982291 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3004.250276] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3004.266584] Lustre: lustre-MDT0000: Denying connection for new client 233d9212-169f-4c8e-a145-f186999f76de (at 192.168.202.55@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3004.290524] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3004.316686] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3004.320653] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3004.330821] Lustre: Skipped 2 previous similar messages [ 3004.370784] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3004.378410] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3020.267078] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3020.278371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3020.294559] Lustre: Skipped 5 previous similar messages [ 3033.056442] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3033.155232] Lustre: server umount lustre-OST0000 complete [ 3039.200655] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3043.415842] Lustre: server umount lustre-OST0001 complete [ 3050.464782] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3050.488752] Lustre: Skipped 7 previous similar messages [ 3053.964696] Lustre: server umount lustre-MDT0000 complete [ 3056.097786] LustreError: 96099:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3056.112436] LustreError: 96099:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 43 previous similar messages [ 3057.612653] LustreError: 95340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790982334 with bad export cookie 5496495062768996543 [ 3057.615845] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3057.626693] LustreError: 95340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3058.087391] Lustre: server umount lustre-MDT0001 complete [ 3067.082878] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:05:41 (1790982341) [ 3075.087558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3075.096655] Lustre: *** cfs_fail_loc=197, val=0*** [ 3090.658258] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3095.775497] LustreError: 103565:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 3100.911236] Lustre: *** cfs_fail_loc=197, val=0*** [ 3100.912825] Lustre: Skipped 65 previous similar messages [ 3121.503349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3121.511146] Lustre: Skipped 3 previous similar messages [ 3127.645961] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3135.261321] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3137.831809] Lustre: server umount lustre-OST0000 complete [ 3146.504943] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:07:00 (1790982420) [ 3168.603605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3174.893077] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3185.429358] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3189.458328] LustreError: 105215:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3189.473748] LustreError: 105215:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3189.484431] LustreError: 105215:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3189.738939] Lustre: server umount lustre-MDT0000 complete [ 3221.786609] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3233.209224] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3267.820062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3268.057804] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3268.081116] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3268.166973] Lustre: lustre-MDT0000: new disk, initializing [ 3268.277841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3272.447988] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3282.186576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3282.293736] Lustre: 109392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3282.604159] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3287.083982] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3297.607825] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3304.087789] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3311.266284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3311.456096] Lustre: 111361:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3318.808797] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3321.590979] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3329.316378] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3334.569186] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3334.708489] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3339.438730] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3339.744334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3347.428161] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3347.428419] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3347.438580] LustreError: Skipped 3 previous similar messages [ 3347.441035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3347.441041] Lustre: Skipped 9 previous similar messages [ 3347.464840] Lustre: Skipped 2 previous similar messages [ 3348.524942] Lustre: server umount lustre-OST0000 complete [ 3357.921676] Lustre: server umount lustre-MDT0000 complete [ 3361.717060] LustreError: 109384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790982638 with bad export cookie 5496495062769007141 [ 3361.724702] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3361.731205] LustreError: 109384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3362.074959] Lustre: server umount lustre-MDT0001 complete [ 3369.605276] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:10:44 (1790982644) [ 3379.232879] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 3382.963815] Key type lgssc unregistered [ 3383.239413] LNet: 113846:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3383.253659] LNetError: 113846:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3383.272013] LNet: Removed LNI 192.168.202.155@tcp [ 3384.249253] Key type .llcrypt unregistered [ 3384.258912] Key type ._llcrypt unregistered [ 3403.684605] Key type ._llcrypt registered [ 3403.686522] Key type .llcrypt registered [ 3403.857887] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3422.176768] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3423.198945] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3423.356425] alg: No test for adler32 (adler32-zlib) [ 3424.600086] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 3424.938567] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 3426.611411] Key type lgssc registered [ 3428.032329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3477.021559] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3477.081079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3478.541861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3478.590077] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3478.712947] Lustre: lustre-MDT0000: new disk, initializing [ 3478.873737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3478.895871] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3482.982352] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3494.314538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3494.619256] Lustre: lustre-OST0000: new disk, initializing [ 3494.625316] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3494.633055] Lustre: Skipped 1 previous similar message [ 3494.638489] Lustre: 118034:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3494.707026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3496.341193] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3496.352107] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3496.415221] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3501.162662] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3511.257217] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3511.490924] Lustre: lustre-OST0001: new disk, initializing [ 3511.494802] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3511.504611] Lustre: 118905:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3511.606809] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3518.568837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3518.588329] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3518.634994] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3518.733624] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3528.674094] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3528.692814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3528.693865] Lustre: Skipped 1 previous similar message [ 3528.707176] Lustre: Skipped 1 previous similar message [ 3533.463427] Lustre: server umount lustre-MDT0000 complete [ 3540.055986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3545.375947] LustreError: 117221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790982822 with bad export cookie 16414136215102562371 [ 3545.381416] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3545.384923] LustreError: 117221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3545.441391] Lustre: server umount lustre-OST0000 complete [ 3549.356474] Lustre: server umount lustre-OST0001 complete [ 3559.073368] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 19:13:53 (1790982833) [ 3588.422775] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 19:14:21 (1790982861) [ 3600.869853] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3625.395440] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:14:59 (1790982899) [ 3668.220191] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3679.487507] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3719.860293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3720.158284] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3720.241465] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3720.416824] Lustre: lustre-MDT0000: new disk, initializing [ 3720.549687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3720.579307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3725.587902] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3738.045941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3738.225644] Lustre: 126663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3738.281618] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3738.285322] Lustre: Skipped 1 previous similar message [ 3738.365710] Lustre: lustre-MDT0001: new disk, initializing [ 3738.463280] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3738.494125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3738.500272] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3743.431205] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3752.967638] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3760.646169] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3769.524688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3770.090286] Lustre: lustre-OST0000: new disk, initializing [ 3770.105302] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3770.130864] Lustre: 128634:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3770.274591] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3778.886854] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3779.198191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3779.213811] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3779.391765] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3791.456181] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3797.488298] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3797.779151] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3803.396643] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3803.654791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3810.283506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3810.284117] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3810.289869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3810.296937] Lustre: Skipped 1 previous similar message [ 3810.308900] LustreError: Skipped 1 previous similar message [ 3811.899830] Lustre: server umount lustre-OST0000 complete [ 3815.393886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3815.400213] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3815.405600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3815.407905] Lustre: Skipped 1 previous similar message [ 3815.433723] Lustre: Skipped 2 previous similar messages [ 3820.514117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3820.537571] Lustre: Skipped 1 previous similar message [ 3821.564051] Lustre: server umount lustre-MDT0000 complete [ 3825.995448] LustreError: 126657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790983102 with bad export cookie 16414136215102563596 [ 3826.009516] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3826.015159] LustreError: 126657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3826.443909] Lustre: server umount lustre-MDT0001 complete [ 3836.635436] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:18:31 (1790983111) [ 3843.464468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3843.799303] LustreError: 131045:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3843.812220] LustreError: 131045:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3843.865540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3848.582641] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3849.185582] LustreError: 131046:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3853.280587] LustreError: 131045:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3857.005398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3861.765865] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3869.375520] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3876.107393] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3883.411595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3883.927932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3883.935715] Lustre: Skipped 1 previous similar message [ 3890.803764] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3899.789505] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3907.634485] Lustre: *** cfs_fail_loc=159, val=0*** [ 3924.099263] Lustre: lustre-MDT0000: Client e37ab0a6-45fe-4a6e-bf2f-0d0a1ed43069 (at 192.168.202.55@tcp) reconnecting [ 3925.619741] Lustre: *** cfs_fail_loc=159, val=0*** [ 3925.625061] Lustre: Skipped 5 previous similar messages [ 3942.027911] Lustre: lustre-MDT0000: Client e37ab0a6-45fe-4a6e-bf2f-0d0a1ed43069 (at 192.168.202.55@tcp) reconnecting [ 3948.515729] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3948.527973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3948.528041] Lustre: Skipped 1 previous similar message [ 3953.455329] Lustre: server umount lustre-OST0000 complete [ 3958.768760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3958.778599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3958.780840] Lustre: Skipped 1 previous similar message [ 3958.796446] Lustre: Skipped 2 previous similar messages [ 3963.224154] Lustre: server umount lustre-MDT0000 complete [ 3963.877334] LustreError: 131046:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3963.902788] LustreError: 131046:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3967.332926] LustreError: 131027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790983244 with bad export cookie 16414136215102564478 [ 3967.335246] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3967.347153] LustreError: 131027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3967.717943] Lustre: server umount lustre-MDT0001 complete [ 3980.403270] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 3983.728980] Key type lgssc unregistered [ 3984.103154] LNet: 134829:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3984.117857] LNetError: 134829:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3984.143715] LNet: Removed LNI 192.168.202.155@tcp [ 3985.372633] Key type .llcrypt unregistered [ 3985.374838] Key type ._llcrypt unregistered [ 4014.558893] Key type ._llcrypt registered [ 4014.560524] Key type .llcrypt registered [ 4014.652227] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4015.692030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4015.877138] alg: No test for adler32 (adler32-zlib) [ 4017.147807] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 4017.445240] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4019.199275] Key type lgssc registered [ 4020.490221] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4029.415352] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4029.446575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4030.900294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4035.490124] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4040.163964] LustreError: 136260:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4040.201305] LustreError: 136260:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4044.612792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4045.088695] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4049.840874] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4058.852032] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4064.909334] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4073.061129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4073.625464] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4075.742538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4080.920144] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4089.187690] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4099.476033] Lustre: *** cfs_fail_loc=159, val=0*** [ 4116.106107] Lustre: lustre-MDT0000: Client fdb5913e-c212-48ab-b437-a9935fffae72 (at 192.168.202.55@tcp) reconnecting [ 4118.671761] Lustre: *** cfs_fail_loc=159, val=0*** [ 4135.027399] Lustre: lustre-MDT0001: Client fdb5913e-c212-48ab-b437-a9935fffae72 (at 192.168.202.55@tcp) reconnecting [ 4137.017715] Lustre: *** cfs_fail_loc=159, val=0*** [ 4137.029065] Lustre: Skipped 3 previous similar messages [ 4152.440407] Lustre: lustre-MDT0001: Client fdb5913e-c212-48ab-b437-a9935fffae72 (at 192.168.202.55@tcp) reconnecting [ 4169.341806] Lustre: 139321:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4171.798554] Lustre: *** cfs_fail_loc=159, val=0*** [ 4171.807684] Lustre: Skipped 4 previous similar messages [ 4187.265156] Lustre: lustre-MDT0000: Client a230cfd7-d776-4bc3-b772-3ed14903c424 (at 192.168.202.55@tcp) reconnecting [ 4188.861980] Lustre: *** cfs_fail_loc=159, val=0*** [ 4188.871368] Lustre: Skipped 14 previous similar messages [ 4204.663445] Lustre: lustre-MDT0000: Client a230cfd7-d776-4bc3-b772-3ed14903c424 (at 192.168.202.55@tcp) reconnecting [ 4206.505202] Lustre: 139613:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4211.174303] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4211.177784] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4211.186667] LustreError: Skipped 1 previous similar message [ 4211.193197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4213.217483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4213.228368] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4213.239098] LustreError: Skipped 6 previous similar messages [ 4213.244318] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4217.382908] Lustre: server umount lustre-OST0000 complete [ 4223.457403] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4223.470655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4223.474586] Lustre: Skipped 1 previous similar message [ 4223.490933] Lustre: Skipped 2 previous similar messages [ 4227.091675] Lustre: server umount lustre-MDT0000 complete [ 4228.581044] LustreError: 136988:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4228.620974] LustreError: 136988:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4231.919482] LustreError: 139441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790983508 with bad export cookie 16069399177532682611 [ 4231.922713] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4231.928631] LustreError: 139441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4232.414391] Lustre: server umount lustre-MDT0001 complete [ 4244.440523] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4247.371578] Key type lgssc unregistered [ 4247.668946] LNet: 140662:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4247.678881] LNetError: 140662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4247.691761] LNet: Removed LNI 192.168.202.155@tcp [ 4248.211898] Key type .llcrypt unregistered [ 4248.214225] Key type ._llcrypt unregistered [ 4274.143326] Key type ._llcrypt registered [ 4274.148787] Key type .llcrypt registered [ 4274.255850] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4275.109308] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4275.291379] alg: No test for adler32 (adler32-zlib) [ 4276.560575] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 4276.946330] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4278.663170] Key type lgssc registered [ 4279.995595] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4288.574678] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4288.628175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4290.280568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4296.250995] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4300.780164] LustreError: 142093:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4300.809049] LustreError: 142093:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4305.818162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4305.895421] LustreError: 142094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4306.256831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4311.261645] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4319.287333] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4325.026888] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4332.339529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4332.735133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4335.862840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4339.992237] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4347.726039] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4351.165172] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4357.780728] Lustre: 144676:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4360.713502] Lustre: 144724:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4362.076730] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4363.009416] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4366.913576] Lustre: 144870:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4367.829333] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4368.741405] Lustre: 144968:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4377.056142] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4377.075926] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.087426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4377.106605] Lustre: Skipped 1 previous similar message [ 4382.196477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4382.211017] Lustre: Skipped 2 previous similar messages [ 4387.300928] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4387.304969] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4387.321569] Lustre: Skipped 1 previous similar message [ 4387.519110] Lustre: server umount lustre-OST0000 complete [ 4392.422777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4392.425639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4392.440192] Lustre: Skipped 1 previous similar message [ 4392.451357] Lustre: Skipped 1 previous similar message [ 4397.540272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4397.811321] Lustre: server umount lustre-MDT0000 complete [ 4401.861793] LustreError: 142073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790983678 with bad export cookie 15588414967400583993 [ 4401.871535] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4402.286794] Lustre: server umount lustre-MDT0001 complete [ 4414.440980] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4417.663949] Key type lgssc unregistered [ 4418.019890] LNet: 146030:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4418.038391] LNetError: 146030:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4418.066953] LNet: Removed LNI 192.168.202.155@tcp [ 4419.259128] Key type .llcrypt unregistered [ 4419.261661] Key type ._llcrypt unregistered [ 4446.363657] Key type ._llcrypt registered [ 4446.366655] Key type .llcrypt registered [ 4446.508777] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4447.420033] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4447.473526] alg: No test for adler32 (adler32-zlib) [ 4448.802900] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 4449.121471] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4450.816571] Key type lgssc registered [ 4452.329407] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4460.089835] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4460.119294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4461.673152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4466.479808] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4471.274955] LustreError: 147461:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4471.302541] LustreError: 147461:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4475.981988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4476.443742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4481.893829] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4490.594484] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4497.054153] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4503.793144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4504.179332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4506.248844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4511.224382] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4520.104657] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4521.602488] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4526.005572] Lustre: *** cfs_fail_loc=159, val=0*** [ 4541.585520] Lustre: lustre-MDT0000: Client e93e20d9-512f-4281-af50-2c0e7f4be676 (at 192.168.202.55@tcp) reconnecting [ 4547.051665] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4547.059101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4547.075654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4547.553610] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4547.578758] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4547.607525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4552.677994] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4557.818689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4557.825891] Lustre: Skipped 2 previous similar messages [ 4561.375174] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4561.585497] Lustre: server umount lustre-OST0000 complete [ 4568.033440] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4568.056605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4568.064284] Lustre: Skipped 1 previous similar message [ 4572.101732] Lustre: server umount lustre-MDT0000 complete [ 4573.161026] LustreError: 147458:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4573.196762] LustreError: 147458:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4576.655923] LustreError: 148205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790983853 with bad export cookie 6357073851412696743 [ 4576.658628] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4576.674768] LustreError: 148205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4576.988367] Lustre: server umount lustre-MDT0001 complete [ 4591.114173] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4595.846873] Key type lgssc unregistered [ 4596.124652] LNet: 151129:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4596.130470] LNetError: 151129:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4596.148159] LNet: Removed LNI 192.168.202.155@tcp [ 4597.576368] Key type .llcrypt unregistered [ 4597.586768] Key type ._llcrypt unregistered [ 4631.341391] Key type ._llcrypt registered [ 4631.349646] Key type .llcrypt registered [ 4631.533364] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4633.329220] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4633.381056] alg: No test for adler32 (adler32-zlib) [ 4634.543471] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 4634.748729] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4636.551442] Key type lgssc registered [ 4638.590935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4648.240777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4648.285167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4650.035247] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4654.942726] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4659.179688] LustreError: 152558:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4659.210634] LustreError: 152558:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4664.290041] LustreError: 152559:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4664.693187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4665.288228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4670.289704] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4677.599675] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4683.837616] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4691.025100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4691.437120] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4694.537230] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4697.730288] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4705.600168] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4709.870404] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4731.301277] Lustre: lustre-MDT0001: evicting 7523570f-154c-47da-87b5-d430fa419402 (at 192.168.202.55@tcp) by administrative request [ 4731.310137] Lustre: Skipped 1 previous similar message [ 4761.057614] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4761.058960] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4761.077645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4761.106254] Lustre: Skipped 1 previous similar message [ 4766.180033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4766.192049] Lustre: Skipped 2 previous similar messages [ 4771.302813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4771.317148] Lustre: Skipped 1 previous similar message [ 4774.367145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4774.514334] Lustre: server umount lustre-OST0000 complete [ 4781.536797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4781.541721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4781.575412] Lustre: Skipped 1 previous similar message [ 4781.592496] Lustre: Skipped 1 previous similar message [ 4786.658811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4786.669282] Lustre: Skipped 1 previous similar message [ 4793.312497] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4793.508866] Lustre: server umount lustre-MDT0000 complete [ 4796.900573] LustreError: 152558:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4796.920683] LustreError: 152558:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4797.686165] LustreError: 155882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790984074 with bad export cookie 8087352782819283307 [ 4797.691202] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4797.693261] LustreError: 155882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4797.939890] Lustre: server umount lustre-MDT0001 complete [ 4811.484742] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4815.148557] Key type lgssc unregistered [ 4815.497545] LNet: 156636:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4815.512466] LNetError: 156636:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4815.534075] LNet: Removed LNI 192.168.202.155@tcp [ 4816.797951] Key type .llcrypt unregistered [ 4816.800202] Key type ._llcrypt unregistered [ 4893.851962] Key type ._llcrypt registered [ 4893.854435] Key type .llcrypt registered [ 4893.937750] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4895.480192] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4895.521171] alg: No test for adler32 (adler32-zlib) [ 4896.675685] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 4896.951570] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4898.719362] Key type lgssc registered [ 4899.753091] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4908.071728] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4908.086220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4909.501652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4914.701927] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4919.776970] LustreError: 159148:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4919.806589] LustreError: 159148:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4922.754071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4923.303068] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4928.033718] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4936.297272] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4942.616396] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4949.483550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4949.851204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4954.997100] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4956.801672] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4963.979874] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4968.309968] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4980.705041] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4980.718156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4980.745796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4981.115091] Lustre: server umount lustre-OST0000 complete [ 4988.905548] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4988.939592] Lustre: Skipped 1 previous similar message [ 4988.943772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4988.947915] Lustre: Skipped 1 previous similar message [ 4992.080910] Lustre: server umount lustre-MDT0000 complete [ 4994.018988] LustreError: 159878:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4994.056613] LustreError: 159878:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4996.863721] LustreError: 159127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790984273 with bad export cookie 3594560572533783900 [ 4996.887945] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4996.893845] LustreError: 159127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5006.601659] Lustre: server umount lustre-MDT0001 complete [ 5019.994337] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5022.933373] Key type lgssc unregistered [ 5023.230956] LNet: 162712:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5023.239486] LNetError: 162712:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5023.259491] LNet: Removed LNI 192.168.202.155@tcp [ 5024.172991] Key type .llcrypt unregistered [ 5024.176073] Key type ._llcrypt unregistered [ 5054.350483] Key type ._llcrypt registered [ 5054.355124] Key type .llcrypt registered [ 5054.643944] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5056.961951] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5057.077244] alg: No test for adler32 (adler32-zlib) [ 5058.477959] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 5058.839387] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5060.619238] Key type lgssc registered [ 5062.540815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5080.842277] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:39:15 (1790984355) [ 5101.218497] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 5114.438595] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5155.268996] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5155.326833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5156.650154] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5156.680496] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5156.808520] Lustre: lustre-MDT0000: new disk, initializing [ 5156.900345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5156.919654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5163.521445] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5175.737931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5175.884800] Lustre: 166993:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5175.981303] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5175.989256] Lustre: Skipped 1 previous similar message [ 5176.154892] Lustre: lustre-MDT0001: new disk, initializing [ 5176.361814] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5176.418706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5176.432743] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5182.277119] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5192.885375] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5199.100053] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5207.026745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5207.373151] Lustre: lustre-OST0000: new disk, initializing [ 5207.383258] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5207.397287] Lustre: 168963:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5207.533064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5213.797039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5213.807390] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5213.926405] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5214.848869] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5224.491371] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5239.780910] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5239.781942] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5239.800932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5239.811374] LustreError: Skipped 1 previous similar message [ 5239.864225] Lustre: Skipped 1 previous similar message [ 5240.899669] Lustre: server umount lustre-OST0000 complete [ 5247.969072] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5247.981947] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5248.018195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5248.037790] Lustre: Skipped 1 previous similar message [ 5250.015924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5250.031189] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5251.455427] Lustre: server umount lustre-MDT0000 complete [ 5256.253907] LustreError: 166986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790984533 with bad export cookie 16982908879995683103 [ 5256.259836] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5256.268854] LustreError: 166986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5256.643893] Lustre: server umount lustre-MDT0001 complete [ 5270.203506] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5273.730108] Key type lgssc unregistered [ 5274.101982] LNet: 170869:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5274.114795] LNetError: 170869:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5274.139395] LNet: Removed LNI 192.168.202.155@tcp [ 5275.109562] Key type .llcrypt unregistered [ 5275.111907] Key type ._llcrypt unregistered [ 5301.952838] Key type ._llcrypt registered [ 5301.954873] Key type .llcrypt registered [ 5302.059558] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5303.418656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5303.484630] alg: No test for adler32 (adler32-zlib) [ 5304.511577] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 5304.972236] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5306.768889] Key type lgssc registered [ 5308.562540] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5317.108714] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5317.131619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5318.569836] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5324.541294] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5328.866638] LustreError: 172305:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5328.930412] LustreError: 172305:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5333.989047] LustreError: 172304:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5337.293380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5337.692456] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5341.253484] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5351.011748] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5355.664668] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5362.556135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5362.812456] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5363.825544] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5370.003799] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5377.854698] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5388.199389] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5388.230214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5388.263296] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5388.270139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.282044] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5389.427293] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5389.461284] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5389.485985] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5389.504297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.524801] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5390.575970] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5390.596534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5390.604475] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5390.607893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.619302] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5391.677257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.874457] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5392.879458] Lustre: Skipped 1 previous similar message [ 5392.905451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5392.915085] Lustre: Skipped 1 previous similar message [ 5392.924152] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5392.929711] Lustre: Skipped 1 previous similar message [ 5392.932995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.944705] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5392.954651] Lustre: Skipped 1 previous similar message [ 5394.123959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.204938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.320822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.201462] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5397.204950] Lustre: Skipped 3 previous similar messages [ 5397.216244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5397.221328] Lustre: Skipped 3 previous similar messages [ 5397.224658] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5397.227453] Lustre: Skipped 3 previous similar messages [ 5397.231115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.242209] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5397.249822] Lustre: Skipped 3 previous similar messages [ 5398.325734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.481552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.541808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.599605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.647347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.560171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.484200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.338520] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5405.350873] Lustre: Skipped 7 previous similar messages [ 5405.386199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.411692] Lustre: Skipped 7 previous similar messages [ 5405.423150] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5405.430334] Lustre: Skipped 7 previous similar messages [ 5405.435140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.450364] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5405.453214] Lustre: Skipped 7 previous similar messages [ 5406.438529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.494346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.616467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.659294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.502780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.563330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.560640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.607196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.571273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.560190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.454594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.667577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.575885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.860594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.887165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.008673] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5422.014742] Lustre: Skipped 15 previous similar messages [ 5422.033481] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5422.044292] Lustre: Skipped 15 previous similar messages [ 5422.049840] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5422.056521] Lustre: Skipped 15 previous similar messages [ 5422.060738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.073391] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5422.084580] Lustre: Skipped 15 previous similar messages [ 5422.822438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.823473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.855231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.103529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.280579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.150442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.217897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.474261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.476175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.561860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.551943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.681685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.761596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.736856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.832274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.755456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.755582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.814445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.006087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.055187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.138695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.343071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.391682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.350660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.370553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.206487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.223586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.394627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.410932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.380041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.216452] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5454.220371] Lustre: Skipped 30 previous similar messages [ 5454.236161] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5454.242981] Lustre: Skipped 30 previous similar messages [ 5454.246137] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5454.248633] Lustre: Skipped 30 previous similar messages [ 5454.251444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.257193] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5454.264829] Lustre: Skipped 30 previous similar messages [ 5455.348025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.400047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.567940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.763971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.113878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.175806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.116407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.912645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.898699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.010899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.863705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.982346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.040365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.281752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.491729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.487687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.557813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.753646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.834883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.670225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.906955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.910268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.040636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.023716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.944347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.765201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.721127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.588754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.569045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.418773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.285556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.363451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.220093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.078349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.260160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.608106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.688909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.894069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.914570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.318246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.378307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.719217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.791760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.848837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.972699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.937185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.354838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.515113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.640987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.943204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.099509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.107094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.067301] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5511.072338] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5511.087154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.047661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.886602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.953135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.872167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.720517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.581992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.716296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.747232] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5518.751345] Lustre: Skipped 60 previous similar messages [ 5518.762530] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5518.771234] Lustre: Skipped 60 previous similar messages [ 5518.777684] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5518.786030] Lustre: Skipped 60 previous similar messages [ 5518.792521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.800291] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5518.805052] Lustre: Skipped 60 previous similar messages [ 5519.761079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.520062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.259154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.192197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.273939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.584673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.022078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.745943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.737945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.842631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.082082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.117446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.343021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.623784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.727261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.971615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.070440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.140956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.099704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.315822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.376646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.382658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.505329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.905393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.275404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.510789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.757848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.690396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.943443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.817109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.402865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.707140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.931032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.041609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.261610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.501849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.706317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.062420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.176575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.242709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.507458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.794737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.043439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.107702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.141973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.107284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.930553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.014982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.078439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.325231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.249853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.320612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.542357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.722293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.839621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.065220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.104606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.334450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.556422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.723273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.086094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.005756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.105468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.260190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.249798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.652311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.720750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.654560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.778741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.004262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.150125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.175188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.456255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.479367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.640546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.548072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.963662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.259456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.710685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.659802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.700043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.857585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.070683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.236205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.239868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.427413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.336817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.322786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.546129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.486971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.531082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.563420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.622892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.780199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.677805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.646958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.821096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.836103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.169077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.367723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.388683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.474910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.564925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.529533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.740357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.897602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.073453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.280213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.326245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.482045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.994741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.297604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.700209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.906856] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5647.912891] Lustre: Skipped 113 previous similar messages [ 5647.960541] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5647.971456] Lustre: Skipped 113 previous similar messages [ 5647.977365] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5647.985570] Lustre: Skipped 113 previous similar messages [ 5647.989929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.003762] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5648.006681] Lustre: Skipped 113 previous similar messages [ 5649.489257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.947108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.749054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.799494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.926500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.282539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.421246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.508417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.513341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.721580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.883494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.971928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.198716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.195875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.331382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.190796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.410947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.430953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.730430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.978814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.090548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.283976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.602456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.840472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.025652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.158299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.234942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.298557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.268958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.180912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.241723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.413892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.522327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.717407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.888035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.234870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.516176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.695082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.534066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.596127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.527784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.783658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.753600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.729238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.803298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.712388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.764603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.734894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.089525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.398228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.367324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.544609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.503216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.728258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.707525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.739215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.810841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.115885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.950877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.177648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.035381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.232525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.178485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.270563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.369083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.466950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.620546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.805051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.722227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.641535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.770254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.980831] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5727.992217] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5728.006987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.025487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.081507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.098918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.548507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.582152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.585140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.743794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.897520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.607438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.716572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.737679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.750971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.707787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.697389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.583135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.461158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.603074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.543221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.737973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.787872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.623986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.648697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.542183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.438984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.430391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.581268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.589839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.692806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.966825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.057410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.337314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.396543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.303350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.325682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.420351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.661714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.730534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.939362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.100445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.082338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.097598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.283321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.547861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.728370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.838429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.840579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.912560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.776547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.833652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.934474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.920490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.042835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.818189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.736457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.798573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.047756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.218667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.207764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.424196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.445373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.607755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.572730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.624444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.685973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.763532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.724783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.772299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.773251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.885084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.026465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.005073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.969139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.754491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.570206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.393382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.455390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.465480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.613025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.887245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.324229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.476840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.702479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.725296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.913256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.287068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.291908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.477224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.571687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.979931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.900509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.324703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.784760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.977140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.282154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.536221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.526396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.524669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.602446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.885984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.043265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.395669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.592939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.866606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.077407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.268753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.381345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.244884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.393911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.619967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.834936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.824954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.736773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.931371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.015056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.896995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.935917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.878489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.027244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.263733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.404229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.495847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.516947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.721036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.717273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.714289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.684396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.655651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.587831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.680190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.609650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.706488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.607484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.664106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.661209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.795670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.720706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.821813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.841685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.882749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.830231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.813559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.825352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.920225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.799870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.005319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.525742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.730458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.809938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.010715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.342432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.529656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.688149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.952718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.189370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.341250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.320082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.436243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.526053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.755259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.992724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.993452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.982482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5903.996182] Lustre: Skipped 233 previous similar messages [ 5904.015620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5904.030728] Lustre: Skipped 233 previous similar messages [ 5904.038377] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5904.042356] Lustre: Skipped 233 previous similar messages [ 5904.045673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.061761] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5904.071992] Lustre: Skipped 233 previous similar messages [ 5905.208665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.271106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.482322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.654757] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5908.670427] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5908.675501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.918216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.939303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.894651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.827097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.996385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.930350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.963786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.201349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.311465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.383419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.406751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.463359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.380194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.425069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.390885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.385796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.298027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.262491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.322611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.299625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.376337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.589464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.482938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.545420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.849464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.008176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.932507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.016694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.896352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.742359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.468588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.352506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.153859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.125676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.931893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.747728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.629698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.455980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.464158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.348405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.418883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.306740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.335597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.123411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.087792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.955495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.055519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.917571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.046854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.114363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.027895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.946771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.775695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.768595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.573750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.607592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.593716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.632284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.539584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.618935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.546382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.529911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.311828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.386575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.155281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.978405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.811554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.719475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.695654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.750495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.512559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.476968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.422476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.576806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.561633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.395292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.223614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.042763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.950301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.821491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.775765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.589883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.463447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.188309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.122421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.072909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.066158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.826824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.740627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.631538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.464367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.383073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.328633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.265297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.249979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.208395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.885621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.729255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.461742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.234040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.026397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.044357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.773949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.764865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.672349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.717548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.718332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.815818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.786376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.854797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.878526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.789226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.864336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.830273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.934471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.866420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.947215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.910885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.803471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.779204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.872331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.790449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.796682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.890376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.923727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.857168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.762096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.765243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.778743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.714918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.753145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.835169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.831844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.694925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.595204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.374203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.254077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.064793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.859700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.962601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.039739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.895128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.789623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.575757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.628342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.374308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.211238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.984191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.893612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.932219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.953991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.677744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.674734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.522537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.402735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.358858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.531582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.769189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.945190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.964121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.095748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.100843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.257622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.462794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.246488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.223392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.164802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.092721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.095664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.390619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.434303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.565230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.963438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.848438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.860107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.803676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.807490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.925870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.690477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.632473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.569874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.483436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.516191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.535637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.493131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.648878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.433617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.597185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.449443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.434592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.395275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.301910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.309770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.247615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.311336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.282645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.121397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.958084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.730808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.763607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.565782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.294342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.989219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.853233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.712581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.510898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.386485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.250282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.048411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.866648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.618870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.460382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.231863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.969434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.930089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.750447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.668817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.578703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.625436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.418890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.262977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.308261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.247675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.255242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.279287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.190080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.141861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.095342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.996330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.109581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.912169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.716375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.633724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.424283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.444971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.759592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.721612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.943917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.010165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.877037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.110715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.046671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.261192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.163330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.311873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.288376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.294852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.307674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.233377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.186668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.469964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.426686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.474808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.465471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.444162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.484153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.242617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.218982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.125261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.058225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.972323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.970762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.761367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.754842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.694346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.675708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.577439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.421105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.706492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.652062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.518563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.510025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.590208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.490575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.533903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.536848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.738522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.717654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.790640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.837560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.714554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.614370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.760211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.677663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.536564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.444212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.536544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.411583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.389614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.106994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.054801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.015480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.849294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.862037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.781506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.843423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.583283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.483382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.272796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.145124] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6195.156070] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6195.170687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.128510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.073677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.926559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.031097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.100250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.226593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.311159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.327623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.377405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.508031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.470822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.526165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.351342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.368651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.361742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.300401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.252433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.245038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.282355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.154084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.107957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.006223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.891328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.750045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.807497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.678725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.634412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.572056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.608852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.526356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.578929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.592413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.704302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.797122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.874365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.721602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.648246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.663658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.553346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.388657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.426092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.286659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.377940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.266354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.131689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.923869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.686553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.726953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.556774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.312183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.290664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.315328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.245301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.292592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.266908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.502704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.730398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.846321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.234132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.306886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.342189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.283331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.272355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.443442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.602426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.720749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.682721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.584304] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6262.589196] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6262.593831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.199972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.864757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.621733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.392310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.141214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.936056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.494802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.249724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.096767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.059081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.077514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.068252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.072315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.234500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.923362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.022087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.851712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.960679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.947827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.874103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.758895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.871756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.851456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.972155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.859511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.055736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.837884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.819066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.682163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.596792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.637674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.664552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.875203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.699481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.608539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.514931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.421832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.374944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.265952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.190424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.003661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.979393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.077931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.072921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.032141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.067765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.042647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.925755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.669843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.589965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.445861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.159040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.956449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.739467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.776767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.785638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.707169] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6314.716800] LustreError: 172324:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6314.720592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.731330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.919899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.885947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.838150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.773078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.841523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.853462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.798811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.809949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.499137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.531789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.492443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.395139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.118798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.981544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.852150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.953486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.851663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.990171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.022472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.951652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.658322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.614652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.469343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.347990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.406971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.612851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.714061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.541365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.517703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.536351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.731183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.490975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.698602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.126246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.689167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.559427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.714731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.806533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.294770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.583605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.685853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.910932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.140191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.173388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.193117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.483243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.676190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.861233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.673550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.507406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.259385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.438527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.533787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.496707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.564875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.407970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.539687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.780500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.837113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.929854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.023464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.149340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.923811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.850731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.046974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.897371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.083609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.289523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.438643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.560709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.565071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.638295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.725404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.809412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.764656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.902285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.981828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.267953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.144398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.142536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.023948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.750815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.467544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.421366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.216684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.289902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.350821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.758718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.435490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.760706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.756223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.944538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.906903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.274925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.217765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.323728] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6416.327325] Lustre: Skipped 525 previous similar messages [ 6416.340631] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6416.346964] Lustre: Skipped 525 previous similar messages [ 6416.349128] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6416.351773] Lustre: Skipped 525 previous similar messages [ 6416.353402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.359288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6416.362592] Lustre: Skipped 525 previous similar messages [ 6417.434916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.422232] LustreError: 174127:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6774.433338] LustreError: 172324:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7213.043379] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7213.045617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7213.062470] LustreError: Skipped 2 previous similar messages [ 7213.094329] Lustre: Skipped 1 previous similar message [ 7213.109351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7214.247970] Lustre: server umount lustre-OST0000 complete [ 7221.734380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7221.747555] Lustre: Skipped 1 previous similar message [ 7221.763183] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7221.785727] LustreError: Skipped 5 previous similar messages [ 7225.413091] Lustre: server umount lustre-MDT0000 complete [ 7226.856514] LustreError: 173362:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7226.888906] LustreError: 173362:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7230.951892] Lustre: 171604:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790986491/real 1790986491] req@00000000cec880e5 x1877983455516032/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1790986507 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7231.019114] LustreError: 172299:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7231.053109] LustreError: 172299:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7231.222227] LustreError: 172286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790986508 with bad export cookie 9077364878612866793 [ 7231.247569] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7231.696797] Lustre: server umount lustre-MDT0001 complete [ 7244.342757] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7247.613763] Key type lgssc unregistered [ 7247.948794] LNet: 227541:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7247.959030] LNetError: 227541:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7248.999295] LNet: Removed LNI 192.168.202.155@tcp [ 7250.499985] Key type .llcrypt unregistered [ 7250.502804] Key type ._llcrypt unregistered [ 7284.264047] Key type ._llcrypt registered [ 7284.268533] Key type .llcrypt registered [ 7284.389580] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7285.659290] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7285.848294] alg: No test for adler32 (adler32-zlib) [ 7287.083748] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 7287.537117] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7289.287272] Key type lgssc registered [ 7291.089793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7301.510193] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7301.566122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7303.146659] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7308.467225] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7312.373873] LustreError: 228972:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7312.414379] LustreError: 228972:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7317.473199] LustreError: 228973:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7318.762530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7319.116237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7323.692129] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7332.842531] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7339.945622] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7348.302404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7348.743662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7357.294442] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7366.896582] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7372.278310] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7376.248316] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7378.156650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7378.171924] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7378.180942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.191370] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7378.191861] LustreError: 228992:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7383.149747] Lustre: lustre-OST0000: Not available for connect from 192.168.202.55@tcp (stopping) [ 7383.534854] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7383.568862] Lustre: Skipped 1 previous similar message [ 7387.020687] Lustre: server umount lustre-OST0000 complete [ 7398.223184] LustreError: 228992:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7423.273766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7423.457733] LustreError: 232183:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7423.608639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7429.106602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.124822] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.132168] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7429.151107] Lustre: Skipped 1 previous similar message [ 7431.380750] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7439.839677] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7441.737719] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7447.956840] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8eda20d33800.ost_server_uuid 50 [ 7449.597374] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8eda20d33800.ost_server_uuid in IDLE state after 0 sec [ 7454.698275] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7454.701895] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7454.737446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7454.743747] Lustre: Skipped 3 previous similar messages [ 7456.085626] Lustre: server umount lustre-OST0000 complete [ 7459.809910] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7459.813709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7459.817822] Lustre: Skipped 2 previous similar messages [ 7459.825861] Lustre: Skipped 1 previous similar message [ 7464.928773] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7464.934643] Lustre: Skipped 1 previous similar message [ 7470.049617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7470.056980] Lustre: Skipped 1 previous similar message [ 7474.143410] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7474.441827] Lustre: server umount lustre-MDT0000 complete [ 7478.129716] LustreError: 228953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790986754 with bad export cookie 6608118222872478501 [ 7478.130477] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7478.137424] LustreError: 228953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7478.462252] Lustre: server umount lustre-MDT0001 complete [ 7490.918199] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7493.911227] Key type lgssc unregistered [ 7494.450661] LNet: 234184:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7494.467736] LNetError: 234184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7494.503459] LNet: Removed LNI 192.168.202.155@tcp [ 7495.542848] Key type .llcrypt unregistered [ 7495.546076] Key type ._llcrypt unregistered [ 7530.516968] Key type ._llcrypt registered [ 7530.519835] Key type .llcrypt registered [ 7530.666568] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7532.405529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7532.418534] alg: No test for adler32 (adler32-zlib) [ 7533.506260] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 7533.841336] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7535.658902] Key type lgssc registered [ 7537.106077] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7547.837343] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7547.854975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7549.558552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7555.936916] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7559.650736] LustreError: 235930:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7559.666894] LustreError: 235930:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7564.770724] LustreError: 235931:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7566.837832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7567.282945] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7571.847474] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7574.871291] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7581.641659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7581.884584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7587.897298] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7594.230947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7596.187333] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7596.318660] Lustre: lustre-OST0001: new disk, initializing [ 7596.330653] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7596.335375] Lustre: 238239:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7596.411666] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7602.341407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7602.352388] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7602.499444] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7602.881651] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7614.961362] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7624.249463] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7635.979288] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7635.985615] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7635.996692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7638.498343] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7638.500685] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7638.524399] Lustre: Skipped 2 previous similar messages [ 7638.556450] Lustre: Skipped 2 previous similar messages [ 7643.624631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7643.629402] Lustre: Skipped 4 previous similar messages [ 7648.752332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7648.780643] Lustre: Skipped 3 previous similar messages [ 7650.271174] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7650.970873] Lustre: server umount lustre-MDT0000 complete [ 7653.863764] LustreError: 235931:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7653.901169] LustreError: 235931:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7655.910129] LustreError: 235913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790986932 with bad export cookie 1029946477781933809 [ 7655.915290] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7655.915873] LustreError: 236667:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7655.927970] LustreError: 235913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7656.481166] Lustre: server umount lustre-MDT0001 complete [ 7672.206366] Lustre: server umount lustre-OST0000 complete [ 7674.338442] Lustre: 235183:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790986935/real 1790986935] req@ffff904e257ca300 x1877985785382912/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790986951 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7674.367469] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7675.886849] Lustre: server umount lustre-OST0001 complete [ 7699.143086] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7710.670710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7711.166342] LustreError: 241350:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7711.247388] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7715.838980] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7716.322957] LustreError: 241351:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7724.459490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7725.022382] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7730.311776] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7733.476383] Lustre: 242493:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7740.549069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7747.355553] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7749.163910] LustreError: 242846:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7749.173959] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7749.182272] LustreError: 242846:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7756.283534] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7756.374797] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7756.746077] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7756.750426] Lustre: Skipped 1 previous similar message [ 7761.973558] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7763.857199] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7771.464283] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7774.856847] Lustre: 244361:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7786.464592] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7786.470958] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7786.493028] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7797.739222] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7797.759275] Lustre: Skipped 11 previous similar messages [ 7800.287373] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7800.504369] Lustre: server umount mylustre-MDT0000 complete [ 7802.865184] LustreError: 242084:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7802.910543] LustreError: 242084:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 7804.541873] LustreError: 241326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987081 with bad export cookie 1029946477781936637 [ 7804.551303] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7804.564440] LustreError: 241326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7805.041603] Lustre: server umount mylustre-MDT0001 complete [ 7819.752297] Lustre: server umount mylustre-OST0000 complete [ 7823.330414] Lustre: 235182:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790987084/real 1790987084] req@ffff904f41860a80 x1877985785446784/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790987100 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7823.341911] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7823.354890] Lustre: Skipped 3 previous similar messages [ 7823.770172] Lustre: server umount mylustre-OST0001 complete [ 7845.870957] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7855.730917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7856.117384] LustreError: 246671:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7856.225722] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7860.738147] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7869.481803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7873.891358] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7876.554322] Lustre: 247812:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7883.081941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7889.572278] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7894.652888] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7897.071679] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7900.412099] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7900.420146] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7902.927744] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7910.508673] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7913.787631] Lustre: 249683:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7921.130715] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7921.140267] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7921.156512] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7927.265745] LustreError: 246671:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7927.281518] LustreError: 246671:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 7929.069766] Lustre: server umount tfs-MDT0000 complete [ 7932.261069] LustreError: 246646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987209 with bad export cookie 1029946477781938814 [ 7932.261637] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7932.267502] LustreError: 246646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7932.635779] Lustre: server umount tfs-MDT0001 complete [ 7942.113849] Lustre: server umount tfs-OST0000 complete [ 7946.664550] Lustre: server umount tfs-OST0001 complete [ 7966.293666] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7978.039506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7978.642377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7978.646336] Lustre: Skipped 3 previous similar messages [ 7983.496950] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7992.363567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7997.372121] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7999.868471] Lustre: 253079:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8005.557569] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8011.819229] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8018.182682] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8020.476560] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8023.952654] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8023.961326] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8027.333675] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8034.664737] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8038.149260] Lustre: 254949:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8044.800189] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:28:39 (1790987319) [ 8064.488228] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8064.499843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8064.518741] Lustre: Skipped 5 previous similar messages [ 8064.526601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8064.538633] Lustre: Skipped 6 previous similar messages [ 8067.082418] Lustre: server umount lustre-MDT0000 complete [ 8068.070673] LustreError: 255319:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8068.088461] LustreError: 255319:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [ 8070.904253] LustreError: 251914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987347 with bad export cookie 1029946477781940732 [ 8070.916620] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8077.545807] Lustre: server umount lustre-MDT0001 complete [ 8087.591662] Lustre: server umount lustre-OST0000 complete [ 8098.001452] Lustre: server umount lustre-OST0001 complete [ 8104.365295] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8114.379671] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8147.846254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8148.067794] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8148.091831] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8148.149443] Lustre: lustre-MDT0000: new disk, initializing [ 8148.244519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8148.247861] Lustre: Skipped 3 previous similar messages [ 8148.261643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8152.291604] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8162.159708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8162.309232] Lustre: 258133:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8162.370277] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8162.373636] Lustre: Skipped 1 previous similar message [ 8162.429974] Lustre: lustre-MDT0001: new disk, initializing [ 8162.578703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8162.590464] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8166.639824] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8176.315494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8176.597467] Lustre: lustre-OST0000: new disk, initializing [ 8176.599840] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8176.605212] Lustre: 259715:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8178.006427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8178.018162] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8178.167553] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8183.069202] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8191.982884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8192.128272] Lustre: lustre-OST0001: new disk, initializing [ 8192.130752] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8192.139125] Lustre: 260586:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8194.083856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8194.093499] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8194.177330] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8197.775163] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8208.209230] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:31:22 (1790987482) [ 8220.567742] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:31:34 (1790987494) [ 8239.585531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8239.588602] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8239.599505] LustreError: Skipped 1 previous similar message [ 8239.611404] Lustre: Skipped 5 previous similar messages [ 8239.616044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8239.632241] Lustre: Skipped 8 previous similar messages [ 8249.311647] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8249.657555] Lustre: server umount lustre-OST0000 complete [ 8254.946632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8259.440958] Lustre: server umount lustre-MDT0000 complete [ 8262.760469] LustreError: 259714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987539 with bad export cookie 1029946477781942510 [ 8262.764244] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8262.768906] LustreError: 259714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8269.057540] Lustre: server umount lustre-MDT0001 complete [ 8279.078694] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8357.369875] Lustre: server umount lustre-OST0001 complete [ 8364.007525] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8380.886981] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8413.498187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8413.733763] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8413.777467] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8413.869387] Lustre: lustre-MDT0000: new disk, initializing [ 8413.980370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8413.983444] Lustre: Skipped 3 previous similar messages [ 8414.008156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8419.107318] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8429.077893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8429.181475] Lustre: 265438:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8429.472315] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8433.338415] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8440.358907] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8444.865247] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8450.494539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8450.688948] Lustre: lustre-OST0000: new disk, initializing [ 8450.694990] Lustre: Skipped 1 previous similar message [ 8450.697461] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8450.700592] Lustre: Skipped 2 previous similar messages [ 8450.704756] Lustre: 267406:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8452.078855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8452.083114] Lustre: Skipped 1 previous similar message [ 8452.085280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8452.149110] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8455.164730] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8462.494851] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8467.438711] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8467.440948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8467.444832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8467.444842] Lustre: Skipped 10 previous similar messages [ 8467.447569] LustreError: Skipped 1 previous similar message [ 8467.473686] Lustre: Skipped 6 previous similar messages [ 8473.185550] Lustre: server umount lustre-OST0000 complete [ 8482.621498] Lustre: server umount lustre-MDT0000 complete [ 8482.788620] LustreError: 265450:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8482.797919] LustreError: 265450:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 14 previous similar messages [ 8485.383748] LustreError: 267405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987762 with bad export cookie 1029946477781943980 [ 8485.386131] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8485.394198] LustreError: 267405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8485.667688] Lustre: server umount lustre-MDT0001 complete [ 8493.885389] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8496.185898] Key type lgssc unregistered [ 8496.407779] LNet: 269166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8496.414443] LNetError: 269166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8496.427064] LNet: Removed LNI 192.168.202.155@tcp [ 8497.045527] Key type .llcrypt unregistered [ 8497.047709] Key type ._llcrypt unregistered [ 8517.936333] Key type ._llcrypt registered [ 8517.938332] Key type .llcrypt registered [ 8518.026590] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8518.985260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8519.070895] alg: No test for adler32 (adler32-zlib) [ 8520.042738] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 8520.278146] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 8521.935208] Key type lgssc registered [ 8522.897955] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8528.898409] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8528.917268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8530.324098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8533.414981] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8539.847595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8540.105235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8543.940435] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8550.141185] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8554.987630] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8561.005065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8561.415412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8566.546797] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8572.726706] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8573.463590] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8576.161144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8576.165130] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8581.358551] Lustre: server umount lustre-OST0000 complete [ 8586.209527] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8586.214654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8586.227704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8586.233308] Lustre: Skipped 1 previous similar message [ 8590.351809] Lustre: server umount lustre-MDT0000 complete [ 8591.854638] LustreError: 270623:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8591.877499] LustreError: 270623:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8593.161105] LustreError: 270602:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790987870 with bad export cookie 8657193521339263690 [ 8593.171584] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8593.401689] Lustre: server umount lustre-MDT0001 complete [ 8601.478167] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8603.296500] Key type lgssc unregistered [ 8603.509766] LNet: 274208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8603.515368] LNetError: 274208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8604.592467] LNet: Removed LNI 192.168.202.155@tcp [ 8605.271582] Key type .llcrypt unregistered [ 8605.274249] Key type ._llcrypt unregistered [ 8617.569467] Key type ._llcrypt registered [ 8617.571193] Key type .llcrypt registered [ 8617.640192] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8618.346371] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8618.444053] alg: No test for adler32 (adler32-zlib) [ 8619.434465] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 8619.579968] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 8621.191703] Key type lgssc registered [ 8622.113639] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8628.822883] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8628.845668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8644.319394] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8649.440053] LustreError: 275398:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 8676.063230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8681.067485] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8686.446532] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8688.051460] Lustre: server umount lustre-OST0000 complete [ 8700.762992] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8702.784459] Key type lgssc unregistered [ 8703.042673] LNet: 277029:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8703.052255] LNetError: 277029:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8703.086860] LNet: Removed LNI 192.168.202.155@tcp [ 8703.786873] Key type .llcrypt unregistered [ 8703.789438] Key type ._llcrypt unregistered [ 8734.278304] Key type ._llcrypt registered [ 8734.280252] Key type .llcrypt registered [ 8734.345330] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8735.187751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8735.230929] alg: No test for adler32 (adler32-zlib) [ 8736.305503] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 8736.482411] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 8738.112059] Key type lgssc registered [ 8739.446521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8749.687506] loop0: detected capacity change from 0 to 204800000 [ 8749.809753] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8749.821688] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8750.076601] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8750.925836] loop0: detected capacity change from 0 to 204800000 [ 8751.071783] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8751.079888] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8751.324068] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8752.057200] loop0: detected capacity change from 0 to 204800000 [ 8752.223446] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8752.237966] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8752.330779] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8753.153149] loop0: detected capacity change from 0 to 204800000 [ 8753.304411] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8754.315475] loop0: detected capacity change from 0 to 204800000 [ 8754.343328] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8757.980840] loop0: detected capacity change from 0 to 204800000 [ 8758.011529] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8761.006846] loop0: detected capacity change from 0 to 204800000 [ 8761.033514] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8764.489458] loop0: detected capacity change from 0 to 204800000 [ 8764.511745] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8767.525924] loop0: detected capacity change from 0 to 204800000 [ 8767.616926] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8767.654077] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8767.725671] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8767.732366] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8767.741502] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8767.750817] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8767.767315] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8767.778958] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8767.795174] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8767.813403] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8767.822666] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8767.828452] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8767.833761] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8767.838826] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8767.845761] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8767.862544] Lustre: 280437:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8772.763926] Lustre: server umount MGS complete [ 8773.454616] loop0: detected capacity change from 0 to 204800000 [ 8773.530855] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8773.866032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8773.907584] Lustre: lustre-MDT0000: Aborting client recovery [ 8774.632854] loop1: detected capacity change from 0 to 204800000 [ 8774.801565] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8774.851178] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8774.862691] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8774.874689] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8774.883945] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8774.891550] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8774.902297] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8774.908312] Lustre: 280868:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8774.920428] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8775.032155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8775.059051] Lustre: lustre-MDT0001: Aborting client recovery [ 8775.693237] loop2: detected capacity change from 0 to 204800000 [ 8775.811514] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8775.899429] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8775.912249] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8775.922134] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8775.930439] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8775.938158] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8775.947552] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8775.957473] Lustre: 280949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8776.047464] Lustre: Found index 0 for lustre-OST0000, updating log [ 8776.078541] Lustre: 280966:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8776.161285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8776.178772] Lustre: lustre-OST0000: Aborting recovery [ 8776.890352] loop3: detected capacity change from 0 to 204800000 [ 8776.985331] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8777.078938] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8777.084771] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8777.091559] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8777.101915] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8777.110549] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8777.124112] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8777.131539] Lustre: 281031:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8777.176374] Lustre: Found index 1 for lustre-OST0001, updating log [ 8777.222336] Lustre: 281036:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8777.337159] Lustre: lustre-OST0001: Aborting recovery [ 8782.853573] LustreError: 280812:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8783.849204] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8783.851337] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8783.903866] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8783.950828] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8784.879765] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8784.880378] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8785.587790] LustreError: 280779:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2761us: rc = -2 [ 8786.088248] LustreError: 280779:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 759us: rc = -2 [ 8786.105840] LustreError: 280779:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8800.225033] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8800.227099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8800.238920] Lustre: Skipped 2 previous similar messages [ 8800.243934] Lustre: Skipped 1 previous similar message [ 8804.710649] Lustre: server umount lustre-MDT0000 complete [ 8805.303106] LustreError: 280763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790988082 with bad export cookie 10638114034172188664 [ 8805.304086] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8805.313186] LustreError: 280763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8805.348514] LustreError: 280777:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8805.349642] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8805.350376] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8805.350381] Lustre: Skipped 2 previous similar messages [ 8805.356150] LustreError: 280777:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8805.360884] Lustre: Skipped 2 previous similar messages [ 8810.468140] LustreError: 281314:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8810.469699] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8810.483363] LustreError: 281314:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8810.500807] Lustre: Skipped 2 previous similar messages [ 8811.623316] Lustre: server umount lustre-MDT0001 complete [ 8818.535903] Lustre: server umount lustre-OST0000 complete [ 8825.312903] Lustre: server umount lustre-OST0001 complete [ 8831.408854] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 20:41:46 (1790988106) [ 8846.986036] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8882.158072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8882.308178] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8882.326440] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8882.374960] Lustre: lustre-MDT0000: new disk, initializing [ 8882.426368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8882.434729] Lustre: Skipped 1 previous similar message [ 8882.463648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8885.752215] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8891.003618] Lustre: server umount lustre-MDT0000 complete [ 8903.295870] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:42:58 (1790988178) [ 8937.095805] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8950.617187] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8996.385706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8996.691347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8996.766950] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8996.772453] Lustre: Skipped 1 previous similar message [ 8996.855298] Lustre: lustre-MDT0000: new disk, initializing [ 8996.937510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8996.955214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9002.419847] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9013.972659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9014.160706] Lustre: 288135:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9014.291613] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9014.296374] Lustre: Skipped 1 previous similar message [ 9014.425144] Lustre: lustre-MDT0001: new disk, initializing [ 9014.570253] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9014.619631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9014.642559] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9020.075676] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9029.556340] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9036.193919] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9044.012692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9044.503489] Lustre: lustre-OST0000: new disk, initializing [ 9044.509001] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9044.524152] Lustre: 290107:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9044.659114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9050.157804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9050.174025] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9050.326288] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9051.651502] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9061.838514] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9067.422442] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9081.154218] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9087.227969] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9097.526646] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9147.874157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9147.875896] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9147.877634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9147.891268] Lustre: Skipped 1 previous similar message [ 9147.913067] LustreError: Skipped 1 previous similar message [ 9151.702459] Lustre: server umount lustre-OST0000 complete [ 9158.116023] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9158.118319] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9158.141836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9158.141844] Lustre: Skipped 1 previous similar message [ 9158.172965] Lustre: Skipped 1 previous similar message [ 9162.942882] Lustre: server umount lustre-MDT0000 complete [ 9163.232624] LustreError: 289507:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9163.247511] LustreError: 289507:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9166.236364] LustreError: 290105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790988443 with bad export cookie 10638114034172191639 [ 9166.244415] LustreError: 290105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9166.251207] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9166.681978] Lustre: server umount lustre-MDT0001 complete [ 9175.267375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9180.636441] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9185.585720] Lustre: server umount MGS complete [ 9198.440368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9199.041597] LustreError: 294029:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9199.075085] LustreError: 294042:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9199.162336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9203.968255] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9204.196584] LustreError: 294030:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9212.840318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9217.891664] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9225.689838] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9231.345141] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9237.384457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9244.284873] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9251.317853] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9274.849938] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9274.852701] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9274.854424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9274.859569] Lustre: Skipped 1 previous similar message [ 9274.863878] Lustre: Skipped 2 previous similar messages [ 9279.443043] Lustre: server umount lustre-OST0000 complete [ 9285.098570] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9285.103976] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9285.112548] LustreError: Skipped 1 previous similar message [ 9288.939657] Lustre: server umount lustre-MDT0000 complete [ 9290.208441] LustreError: 296660:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9290.222415] LustreError: 296660:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9291.972328] LustreError: 294011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790988568 with bad export cookie 10638114034172193592 [ 9291.973847] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9291.977865] LustreError: 294011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9292.369695] Lustre: server umount lustre-MDT0001 complete [ 9300.967225] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9302.999595] Key type lgssc unregistered [ 9303.188900] LNet: 297972:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9303.192612] LNetError: 297972:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9303.203479] LNet: Removed LNI 192.168.202.155@tcp [ 9303.810181] Key type .llcrypt unregistered [ 9303.812414] Key type ._llcrypt unregistered [ 9329.353249] Key type ._llcrypt registered [ 9329.357532] Key type .llcrypt registered [ 9329.441868] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9338.629966] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9339.243464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9339.260634] alg: No test for adler32 (adler32-zlib) [ 9340.178143] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 9340.342945] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9341.951515] Key type lgssc registered [ 9342.617877] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9363.469889] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9363.485486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9364.630969] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9364.660273] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9364.711232] Lustre: lustre-MDT0000: new disk, initializing [ 9364.754643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9364.768694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9366.910544] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9373.386775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9373.468747] Lustre: 301920:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9373.501532] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9373.504453] Lustre: Skipped 1 previous similar message [ 9373.552671] Lustre: lustre-MDT0001: new disk, initializing [ 9373.633791] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9373.671263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9373.684286] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9376.587555] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9382.988743] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9386.573871] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9391.048679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9391.218065] Lustre: lustre-OST0000: new disk, initializing [ 9391.220945] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9391.224792] Lustre: 303887:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9391.296542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9395.415858] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9401.022701] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9401.367313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9401.372340] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9401.435185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9406.974091] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9412.347648] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9420.013856] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9467.881516] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9467.883981] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9467.896561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9467.904391] Lustre: Skipped 1 previous similar message [ 9467.914715] Lustre: Skipped 1 previous similar message [ 9472.994952] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9473.001252] Lustre: Skipped 1 previous similar message [ 9478.114269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9478.117825] Lustre: Skipped 1 previous similar message [ 9481.183208] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9481.269506] Lustre: server umount lustre-OST0000 complete [ 9486.304401] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9486.310487] LustreError: Skipped 1 previous similar message [ 9486.315473] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9486.326098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9488.356615] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9489.627086] Lustre: server umount lustre-MDT0000 complete [ 9491.827448] LustreError: 301912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790988768 with bad export cookie 1613765059724550865 [ 9491.831699] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9491.843238] LustreError: 301912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9492.102438] Lustre: server umount lustre-MDT0001 complete [ 9497.807467] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9500.529715] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9503.343512] Lustre: server umount MGS complete [ 9509.402941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9509.646153] LustreError: 307763:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9509.678262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9511.821391] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9514.978322] LustreError: 307750:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9514.987867] LustreError: 307750:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9516.507826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9516.773876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9519.503781] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9524.297504] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9527.950454] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9532.234603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9532.439382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9536.044033] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9539.717855] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9560.032756] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9560.045303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9560.065978] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9560.069850] Lustre: Skipped 2 previous similar messages [ 9562.089717] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9564.748598] Lustre: server umount lustre-OST0000 complete [ 9567.200778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9567.209814] Lustre: Skipped 2 previous similar messages [ 9572.320909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9572.325737] Lustre: Skipped 5 previous similar messages [ 9573.138349] Lustre: server umount lustre-MDT0000 complete [ 9575.801315] LustreError: 310474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790988852 with bad export cookie 1613765059724552692 [ 9575.806379] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9575.807958] LustreError: 310474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9576.170052] Lustre: server umount lustre-MDT0001 complete [ 9584.760606] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9586.953702] Key type lgssc unregistered [ 9587.208035] LNet: 311689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9587.212208] LNetError: 311689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9587.231210] LNet: Removed LNI 192.168.202.155@tcp [ 9587.896444] Key type .llcrypt unregistered [ 9587.899551] Key type ._llcrypt unregistered [ 9603.889264] Key type ._llcrypt registered [ 9603.893928] Key type .llcrypt registered [ 9604.014903] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9614.234149] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9614.873607] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9614.907910] alg: No test for adler32 (adler32-zlib) [ 9615.900931] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [ 9616.175907] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9617.807280] Key type lgssc registered [ 9618.703925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9635.543214] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9645.989930] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9646.026870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9647.227447] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9647.253693] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9647.308734] Lustre: lustre-MDT0000: new disk, initializing [ 9647.362127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9647.385594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9650.312353] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9659.394258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9659.609750] Lustre: lustre-OST0000: new disk, initializing [ 9659.612709] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9659.615485] Lustre: Skipped 1 previous similar message [ 9659.619860] Lustre: 315733:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9659.663343] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9663.053908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9663.062314] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9663.110629] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9664.757758] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10118.626897] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10118.637198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10121.865132] Lustre: server umount lustre-MDT0000 complete [10123.955579] LustreError: 314920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790989400 with bad export cookie 8626567065630837653 [10123.959048] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10123.960528] LustreError: 314920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10124.053789] Lustre: server umount lustre-OST0000 complete [10154.400338] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [10171.980731] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10211.969851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10212.190034] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10212.214087] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10212.266520] Lustre: lustre-MDT0000: new disk, initializing [10212.336198] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10212.346595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10216.143210] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10224.871133] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10224.955563] Lustre: 320043:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10225.011942] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10225.014487] Lustre: Skipped 1 previous similar message [10225.065445] Lustre: lustre-MDT0001: new disk, initializing [10225.195250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10225.232850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10225.251624] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10228.697876] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10236.408801] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10240.483111] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10245.662504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10245.867270] Lustre: lustre-OST0000: new disk, initializing [10245.870249] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10245.874735] Lustre: 322009:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10245.951596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10250.790868] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10252.849854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10252.860773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10252.981926] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10259.187423] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10263.129535] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10263.298964] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10266.680597] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10266.847448] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10268.641201] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10268.641988] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10268.646174] LustreError: Skipped 1 previous similar message [10268.654037] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10273.768781] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10273.774774] Lustre: Skipped 1 previous similar message [10274.479746] Lustre: server umount lustre-OST0000 complete [10281.442812] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10281.465793] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10281.484453] Lustre: Skipped 1 previous similar message [10281.504565] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10281.517619] Lustre: Skipped 1 previous similar message [10284.002431] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10286.623275] Lustre: server umount lustre-MDT0000 complete [10292.465876] LustreError: 320036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790989569 with bad export cookie 8626567065633778437 [10292.469732] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10292.483840] LustreError: 320036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10292.919753] Lustre: server umount lustre-MDT0001 complete [10299.428852] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10303.072898] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:06:16 (1790989576) [10312.207868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10312.794032] LustreError: 324322:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10312.826414] LustreError: 324322:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10312.973161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10318.305530] LustreError: 324323:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10318.690831] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10322.477071] LustreError: 324322:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10327.533603] LustreError: 324323:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10328.313188] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10334.003937] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10343.208375] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10350.600618] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10358.999249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10359.646483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10359.648760] Lustre: Skipped 1 previous similar message [10366.304867] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10374.368703] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10381.606497] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10381.885993] Lustre: lustre-OST0001: new disk, initializing [10381.901504] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10381.911700] Lustre: 327158:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10389.638334] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10391.210244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10391.222296] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10391.303243] LustreError: 327587:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10392.358595] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10392.375085] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10393.377972] LustreError: 327587:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10395.441222] LustreError: 327587:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10395.456258] LustreError: 327587:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10399.596378] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10399.607913] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10400.177661] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10400.387391] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10406.685650] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8eda0ae75800.ost_server_uuid 50 [10407.859670] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10407.883055] LustreError: 327594:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10408.331644] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8eda0ae75800.ost_server_uuid in FULL state after 0 sec [10415.208961] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8eda0ae75800.ost_server_uuid 50 [10416.922344] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8eda0ae75800.ost_server_uuid in FULL state after 0 sec [10424.807284] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10434.529956] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10434.536049] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10434.553644] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10434.558808] Lustre: Skipped 2 previous similar messages [10438.904291] Lustre: server umount lustre-OST0001 complete [10440.715392] LustreError: 326146:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10440.726380] LustreError: 326146:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10445.796263] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10445.804742] Lustre: Skipped 1 previous similar message [10445.808857] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10445.810756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10445.813822] LustreError: Skipped 1 previous similar message [10445.820930] Lustre: Skipped 4 previous similar messages [10450.948653] LustreError: 326146:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10450.970050] LustreError: 326146:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [10451.864182] Lustre: server umount lustre-OST0000 complete [10456.561638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10462.673944] Lustre: server umount lustre-MDT0000 complete [10466.749932] LustreError: 327157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790989743 with bad export cookie 8626567065633779326 [10466.762955] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10467.113826] Lustre: server umount lustre-MDT0001 complete [10480.065153] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [10483.112624] Key type lgssc unregistered [10483.503465] LNet: 329844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10483.513720] LNetError: 329844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10484.583669] LNet: Removed LNI 192.168.202.155@tcp [10485.277094] Key type .llcrypt unregistered [10485.278111] Key type ._llcrypt unregistered [10512.987628] Key type ._llcrypt registered [10512.989922] Key type .llcrypt registered [10513.118851] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10514.156580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10514.240416] alg: No test for adler32 (adler32-zlib) [10515.289698] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [10515.657827] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [10517.367253] Key type lgssc registered [10519.064535] Lustre: Echo OBD driver; http://www.lustre.org/ [10527.155571] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10527.190530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10528.629339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10533.080788] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10538.992933] LustreError: 331285:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10539.031517] LustreError: 331285:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10541.402709] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10544.101829] LustreError: 331284:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10548.022877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10548.368184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10552.941622] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10562.552610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10562.929340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10570.550764] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10573.156141] LustreError: 332964:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10573.201464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10578.415047] LustreError: 332963:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10578.463815] LustreError: 332963:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10581.865922] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10583.526568] LustreError: 333398:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10583.575857] LustreError: 333398:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10592.153669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10592.642044] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10597.897757] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10598.758954] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10606.101324] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10614.699280] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10615.023985] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10710.497674] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10710.511062] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10710.527377] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10710.533519] Lustre: Skipped 1 previous similar message [10715.342391] Lustre: server umount lustre-OST0001 complete [10715.620155] LustreError: 332965:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10715.634259] LustreError: 332965:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10720.749218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10720.754646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10720.762209] Lustre: Skipped 2 previous similar messages [10720.773299] Lustre: Skipped 1 previous similar message [10725.187957] Lustre: server umount lustre-OST0000 complete [10730.977221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10730.979404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10730.988788] Lustre: Skipped 1 previous similar message [10734.790207] Lustre: server umount lustre-MDT0000 complete [10736.096518] LustreError: 331285:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10736.113604] LustreError: 331285:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10737.715487] LustreError: 331264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790990014 with bad export cookie 14492733653323185477 [10737.725271] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10737.737409] LustreError: 331264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10737.982945] Lustre: server umount lustre-MDT0001 complete [10744.342407] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [10745.967789] Key type lgssc unregistered [10746.140383] LNet: 336224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10746.147415] LNetError: 336224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10746.161551] LNet: Removed LNI 192.168.202.155@tcp [10746.666689] Key type .llcrypt unregistered [10746.667908] Key type ._llcrypt unregistered [10776.040766] Key type ._llcrypt registered [10776.042393] Key type .llcrypt registered [10776.110187] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10776.951793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10776.987830] alg: No test for adler32 (adler32-zlib) [10777.933258] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [10778.078045] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [10779.695190] Key type lgssc registered [10780.507731] Lustre: Echo OBD driver; http://www.lustre.org/ [10786.767452] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10786.784407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10788.087831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10791.092940] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10797.092057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10797.469433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10801.374861] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10803.202472] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10807.244503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10807.454349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10808.491954] LustreError: 340098:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10808.511028] LustreError: 340098:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10811.975452] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10813.611571] LustreError: 340098:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10813.620691] LustreError: 340098:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10813.627111] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10817.493284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10817.630368] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10821.387415] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10822.660550] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10829.113883] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10834.496329] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10843.110919] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10843.118556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10843.125702] Lustre: Skipped 2 previous similar messages [10843.132430] Lustre: Skipped 3 previous similar messages [10847.608044] Lustre: server umount lustre-MDT0000 complete [10848.232656] LustreError: 339250:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10848.251041] LustreError: 339250:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10849.989522] LustreError: 340017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790990126 with bad export cookie 4793622939660385353 [10849.990186] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10849.996379] LustreError: 340017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10850.230996] Lustre: server umount lustre-MDT0001 complete [10862.610254] Lustre: server umount lustre-OST0000 complete [10874.693256] Lustre: server umount lustre-OST0001 complete [10884.395186] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10891.010820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10891.284596] LustreError: 343772:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10891.357885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10893.494485] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10896.353391] LustreError: 343773:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10897.768788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10900.077233] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10901.555576] Lustre: 344914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10905.570913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10905.764995] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10905.770101] Lustre: Skipped 1 previous similar message [10907.817142] LustreError: 345266:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10909.665213] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10914.525158] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10916.781762] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10916.793035] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10918.570880] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10923.651732] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10926.258319] Lustre: 346780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10928.608225] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10928.614593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10928.619711] Lustre: Skipped 1 previous similar message [10928.622973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10932.194696] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10932.197440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10932.211799] Lustre: Skipped 1 previous similar message [10932.217224] Lustre: Skipped 1 previous similar message [10934.867391] Lustre: server umount lustre-MDT0000 complete [10937.313917] LustreError: 343768:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10937.327663] LustreError: 343768:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [10937.593751] LustreError: 343754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790990214 with bad export cookie 4793622939660387054 [10937.598957] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10937.613816] LustreError: 343754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10937.801198] Lustre: server umount lustre-MDT0001 complete [10949.653834] Lustre: server umount lustre-OST0000 complete [10961.307605] Lustre: server umount lustre-OST0001 complete [10963.304992] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10964.081563] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10964.910121] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:17:20 (1790990240) [10977.560485] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10983.192193] loop0: detected capacity change from 0 to 1099511627776 [10983.420894] SGI XFS with ACLs, security attributes, quota, no debug enabled [10983.424982] *** VALIDATE xfs *** [10983.431814] XFS (loop0): Mounting V5 Filesystem [10983.447361] XFS (loop0): Ending clean mount [10984.200315] loop1: detected capacity change from 0 to 18691697672192 [11026.816927] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [11031.571372] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11051.206611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11051.373836] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11051.386826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11051.447984] Lustre: lustre-MDT0000: new disk, initializing [11051.496444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11051.501766] Lustre: Skipped 1 previous similar message [11051.513776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11054.422646] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11061.174273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11061.240546] Lustre: 351897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11061.273670] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11061.276161] Lustre: Skipped 1 previous similar message [11061.316437] Lustre: lustre-MDT0001: new disk, initializing [11061.385407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11061.392905] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11063.267528] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11068.459466] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11071.649914] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11075.407312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11075.575980] Lustre: lustre-OST0000: new disk, initializing [11075.578852] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11075.581697] Lustre: 353862:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11077.389207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11077.399988] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11077.470963] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11079.141719] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11085.887655] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11088.603396] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11088.738766] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11091.356754] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11091.476198] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11092.962032] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11092.968668] Lustre: Skipped 1 previous similar message [11092.971117] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11092.971489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11092.982192] Lustre: Skipped 3 previous similar messages [11098.089024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11098.100258] Lustre: Skipped 1 previous similar message [11098.696388] Lustre: server umount lustre-OST0000 complete [11102.688412] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11102.701752] LustreError: Skipped 1 previous similar message [11102.705896] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11102.712603] Lustre: Skipped 1 previous similar message [11106.936873] Lustre: server umount lustre-MDT0000 complete [11108.321057] LustreError: 351902:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11108.336934] LustreError: 351902:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11109.473640] LustreError: 351887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790990386 with bad export cookie 4793622939660388601 [11109.487373] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11109.706946] Lustre: server umount lustre-MDT0001 complete [11115.279296] XFS (loop0): Unmounting Filesystem [11117.954481] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:19:52 (1790990392) [11121.975599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11122.245372] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11122.249217] Lustre: Skipped 2 previous similar messages [11124.498954] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11130.029523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11133.038380] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11137.609523] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11140.857427] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11144.977436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11148.682171] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11152.470153] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11160.548224] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11160.549178] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11160.549842] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11160.549846] Lustre: Skipped 3 previous similar messages [11160.569470] Lustre: Skipped 2 previous similar messages [11162.339040] Lustre: server umount lustre-OST0000 complete [11166.273458] Lustre: server umount lustre-MDT0000 complete [11168.429328] LustreError: 356444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790990445 with bad export cookie 4793622939660389483 [11168.433210] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11168.436986] LustreError: 356444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [11168.668831] Lustre: server umount lustre-MDT0001 complete [11175.721530] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [11177.155029] Key type lgssc unregistered [11177.311894] LNet: 360043:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11177.317165] LNetError: 360043:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11177.328725] LNet: Removed LNI 192.168.202.155@tcp [11177.890268] Key type .llcrypt unregistered [11177.892456] Key type ._llcrypt unregistered [11193.212660] Key type ._llcrypt registered [11193.214748] Key type .llcrypt registered [11193.276481] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11193.849207] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11193.859273] alg: No test for adler32 (adler32-zlib) [11194.834280] Lustre: Lustre: Build Version: 2.17.59_2_ge06ed12 [11195.023537] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [11196.663244] Key type lgssc registered [11197.281828] Lustre: Echo OBD driver; http://www.lustre.org/ [11201.366707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11201.378336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11202.631139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11204.977858] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11209.207479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11209.441768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11211.323889] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11215.340887] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11218.793414] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11222.925064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11223.169966] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11226.765846] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11230.561921] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11239.905278] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11239.925397] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11243.077291] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [11243.081372] Lustre: Skipped 1 previous similar message [11245.024944] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11245.033089] Lustre: Skipped 1 previous similar message [11245.715228] Lustre: server umount lustre-MDT0001 complete [11248.189357] LustreError: 363033:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11248.200978] LustreError: 363033:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11250.144312] LustreError: 362340:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11250.150283] LustreError: 362340:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11250.771049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11250.830952] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11250.835646] Lustre: Found index 1 for lustre-MDT0001, updating log [11250.864027] Lustre: 361593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11251.000682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11252.989787] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11256.297243] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11256.301819] LustreError: 365209:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11256.306177] LustreError: 365209:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11256.310885] LustreError: 361595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11256.339309] LustreError: 365215:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [11256.342463] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11256.342463] [11558.372312] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11558.379222] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11558.387536] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11558.394871] Lustre: Skipped 2 previous similar messages [11558.430714] Lustre: Skipped 1 previous similar message [11563.490651] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11563.505898] Lustre: Skipped 2 previous similar messages [11565.128783] Lustre: server umount lustre-MDT0001 complete [11565.623516] LustreError: 366067:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11568.609211] LustreError: 361603:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11573.738565] LustreError: 361602:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11573.771889] LustreError: 361602:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11577.781881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11577.957730] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11577.969178] Lustre: Found index 1 for lustre-MDT0001, updating log [11578.045731] Lustre: 361591:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11578.428798] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11583.472723] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11585.308304] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11587.079563] LustreError: 375873:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11587.093919] LustreError: 375873:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11587.115251] LustreError: 361595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11587.133101] LustreError: 361595:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11587.226872] LustreError: 375883:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11587.236942] LustreError: 375883:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11587.239694] LustreError: 375883:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [11587.258248] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.155@tcp [11587.258248] [11891.290679] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [11895.781051] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11895.784124] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11895.815733] Lustre: Skipped 1 previous similar message [11897.425480] Lustre: server umount lustre-MDT0001 complete [11900.896051] LustreError: 361602:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11900.916050] LustreError: 361602:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11906.819889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11906.883737] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11906.888384] Lustre: Found index 1 for lustre-MDT0001, updating log [11906.918145] Lustre: 361592:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11907.257597] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11911.317600] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11912.702040] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11914.233091] LustreError: 385194:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11914.247080] LustreError: 385194:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11914.265199] LustreError: 361595:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11914.272576] LustreError: 361595:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11914.412228] LustreError: 385197:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11914.447300] LustreError: 385197:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -114 [11914.460823] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11914.460823] [12233.194275] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12233.200296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12233.209657] Lustre: Skipped 2 previous similar messages [12233.224047] Lustre: Skipped 5 previous similar messages [12237.728393] Lustre: server umount lustre-MDT0000 complete [12238.305487] LustreError: 363033:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12238.337483] LustreError: 363033:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [12241.542315] LustreError: 361583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790991518 with bad export cookie 14504531225207157799 [12241.557790] LustreError: 361583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12241.561152] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12243.425714] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12248.031970] Lustre: server umount lustre-MDT0001 complete [12251.838148] Lustre: server umount lustre-OST0000 complete [12261.650738] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [12270.042858] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12302.356458] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:39:36 (1790991576) [12308.426947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12308.769278] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12308.813723] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12308.964247] Lustre: lustre-MDT0000: new disk, initializing [12309.100613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12309.115400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12313.522616] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12323.347193] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12323.462334] Lustre: 396276:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12323.542105] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12323.548751] Lustre: Skipped 1 previous similar message [12323.678066] Lustre: lustre-MDT0001: new disk, initializing [12323.797948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12323.852147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12323.867923] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12327.759748] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12336.831265] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12341.670353] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12346.664984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12346.861168] Lustre: lustre-OST0000: new disk, initializing [12346.863541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12346.867749] Lustre: 398241:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12346.937628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12351.255340] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12355.607853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12355.620770] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12355.711632] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12358.554876] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12364.769365] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12364.776091] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12364.782372] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12364.785645] Lustre: Skipped 2 previous similar messages [12370.596784] Lustre: server umount lustre-MDT0000 complete [12370.914508] LustreError: 396281:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12370.938855] LustreError: 396281:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12373.407835] LustreError: 396268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790991650 with bad export cookie 14504531225207159738 [12373.410035] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12373.414849] LustreError: 397081:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12373.428844] LustreError: 396268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12373.689295] Lustre: server umount lustre-MDT0001 complete [12387.211117] Lustre: server umount lustre-OST0000 complete [12410.852972] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [12419.559655] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12444.960684] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12163 sec ========= 21:41:59 (1790991719) [12446.127626] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:42:00 (1790991720) === [12448.748329] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:42:03 (1790991723) === [12479.449552] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [12481.587890] Key type lgssc unregistered [12481.816921] LNet: 404602:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12481.827882] LNetError: 404602:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12481.851858] LNet: Removed LNI 192.168.202.155@tcp [12482.462178] Key type .llcrypt unregistered [12482.463911] Key type ._llcrypt unregistered